python - @kholidfu
Table of Contents
- 1. Deploy Flask + Nginx + Supervisor di Ubuntu
- 2. Install library lxml di dalam virtualenv
- 3. Catch search engine terms with python, flask and mongo
- 4. Mencegah Cascade di django models
- 5. CSS, JS dan IMG tidak mau load di django-admin
- 6. Setting gunicorn + django + nginx
- 7. Up and Running with django-cookiecutter
- 8. Akses ebay SOAP API dengan Python
- 9. Konfigurasi dan Implementasi PostgreSQL HStoreField di django
- 10. Setting nginx + uwsgi + supervisor + django
- 11. Port di django vagrant
1 Deploy Flask + Nginx + Supervisor di Ubuntu
1.1 Add user di Ubuntu
adduser sopier adduser sopier sudo
1.2 Install needed package
sudo apt-get install build-essential python-dev python-pip nginx emacs24-nox git
1.3 Install virtualenv & supervisor
sudo pip install virtualenv supervisor
1.4 git clone
git clone https://github.com/sopier/example.com
1.5 Setting virtualenv
cd example.com virtualenv . . bin/activate pip install -r requirements.txt pip install uwsgi
1.6 Setting supervisor
Buat berkas baru, misal di ~/supervisord.conf
, dan isikan baris
konfigurasi berikut:
[unix_http_server] file=/tmp/uwsgi.sock ; (the path to the socket file) [supervisord] logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log) logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB) logfile_backups=10 ; (num of main logfile rotation backups;default 10) loglevel=info ; (log level;default info; others: debug,warn,trace) pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid) nodaemon=false ; (start in foreground if true;default false) minfds=1024 ; (min. avail startup file descriptors;default 1024) minprocs=200 [rpcinterface:supervisor] supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface [supervisorctl] logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log) serverurl=unix:///tmp/uwsgi.sock ; use a unix:// URL [program:example.com] command=/home/sopier/example.com/bin/uwsgi --socket /tmp/uwsgi.sock --module run --callable app -H /home/sopier/example.com/ --chdir /home/sopier/example.com/ --chmod-socket=666 --processes=5 directory=/home/sopier/example.com/ autostart=true autorestart=true stdout_logfile=/tmp/example.log redirect_stderr=true stopsignal=QUIT
1.7 Sunting berkas run.py
Berikut ini setting untuk run.py (sesuaikan seperlunya):
#!/usr/bin/env python from app import app if __name__ == "__main__": app.run(debug=False)
1.8 Setting nginx
Berikut ini adalah konfigurasi nginx sederhana tapi works:
server { listen 80; server_name www.example.com; location / { try_files $uri @app; } location @app { include uwsgi_params; uwsgi_pass unix:/tmp/uwsgi.sock; } }
Atau jika Anda ingin multiple site, berikut ini contoh konfigurasinya:
server { listen 80; server_name www.example1.com; location / { try_files $uri @app; } location @app { include uwsgi_params; uwsgi_pass unix:/tmp/uwsgi1.sock; } } server { listen 80; server_name www.example2.com; location / { try_files $uri @app; } location @app { include uwsgi_params; uwsgi_pass unix:/tmp/uwsgi2.sock; } }
1.9 Jalankan supervisor
Terakhir, jalankan supervisor
dengan menjalankan perintah berikut:
sudo supervisord -c ~/supervisord.conf
2 Install library lxml di dalam virtualenv
sudo apt-get install libxml2-dev libxslt-dev
Jika server Anda terlalu kecil spec-nya (gagal di DO yang $5/month,
bisa jadi terjadi error, ini terjadi karena proses compile
membutuhkan memory besar. Untuk menambah memory, kita sementara
dapat menambah swap
.
Caranya adalah sebagai berikut:
# create new partition dd if=/dev/zero of=/swapfile bs=1M count=1024 # set as swap mkswap /swapfile # activate the swap once swapon /swapfile # run every machine reboot (optional) /swapfile swap swap defaults 0 0 # add into /etc/fstab
3 Catch search engine terms with python, flask and mongo
# pip install referer_parser from referer_parser import Referer # dbase import pymongo c = pymongo.Connection() refdb = c['referer'] # ganti "-" => " " pada q t = q.replace("-", " ") # referer tracker try: head = request.headers url = head['Referer'] refobj = Referer(url) # setem ini sudah bersih! setem = refobj.search_term # jika ada setem, input into db if setem: # jika lom ada setem, insert! if refdb.term.find_one({"q": t}) is None: refdb.term.insert({"q": t, "refer": {setem: 1}}) # jika sudah ada, pilihannya ada 2, set atau increment! else: # jika sudah ada setem, increment! if setem in refdb.term.find_one({"q": t})['refer']: refdb.term.update( {"q": t}, {"$inc": {"refer." + str(setem): 1}}, upsert=True ) # jika lom ada setem, set setem: 1! elif setem not in refdb.term.find_one({"q": t})['refer']: refdb.term.update( {"q": t}, {"$set": {"refer." + str(setem): 1}}, upsert=True ) except: pass # sorting term sorted( db.term.find_one({"q": t})['refer'].items(), key=lambda x: x[1], reverse=True )
4 Mencegah Cascade di django models
django, by default menggunakan cascade untuk model ForeignKey, ini artinya jika data child dihapus maka data parent juga ikut terhapus. Untuk mencegah hal ini, kita dapat menambahkan argument berikut:
nama = models.ForeignKey(Guru, on_delete=models.SET_NULL)
Jika setting sudah benar, tandanya ada tanda silang di sebelah field ForeignKey.
5 CSS, JS dan IMG tidak mau load di django-admin
Jika hal ini terjadi, berarti setting nginx Anda perlu dibenahi, yakni dengan menambahkan location static di konfig nginx Anda, misal sebagai berikut:
server { ... location /static/ { alias /home/sopier/mslib/mslib/static/; } ... }
6 Setting gunicorn + django + nginx
Dengan asumsi struktur direktori sebagai berikut:
mslib/ ├── app │ ├── admin.py │ ├── __init__.py │ ├── migrations │ ├── models.py │ ├── tests.py │ ├── urls.py │ ├── views.py ├── manage.py ├── mslib │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ ├── wsgi.py ├── static │ ├── admin │ ├── app │ └── js └── templates ├── 404.html ├── admin └── app
Dan berikut ini setting untuk nginx:
upstream app_server { server 127.0.0.1:8000 fail_timeout=0; } server { listen 80; server_name 159.xxx.xxx.xxx; client_max_body_size 4G; proxy_read_timeout 1200; location / { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_redirect off; proxy_pass http://app_server; if (!-f $request_filename) { proxy_pass http://app_server; break; } } location /static/ { alias /home/sopier/mslib/mslib/static/; } location /media/ { alias /home/sopier/mslib/mslib/collections/; } }
Maka untuk menjalankan gunicorn:
gunicorn --env DJANGO_SETTINGS_MODULE=mslib.settings mslib.wsgi --bind 127.0.0.1:8000
7 Up and Running with django-cookiecutter
# starting django project with cookiecutter cookiecutter https://github.com/pydanny/cookiecutter-django.git # choose Y # It will clone in ~/.cookiecutter # Follow the wizard cd into project_name virtualenv . # edit requirements/base.txt, add/comment unwanted libs pip install -r requirements/local.txt # install gunicorn pip install gunicorn # create database name sudo -u postgres -i createdb project_name # do migration python manage.py migrate python manage.py runserver # creating project/apps # you can create app straightforward in project_name/ dir python manage.py createapp myapp # or # startproject and createapp python manage.py startproject myproject cd myproject python manage.py createapp myapp # I prefer the 1st one # run server with gunicorn gunicorn --env DJANGO_SETTINGS_MODULE=config.settings.local config.wsgi --bind 127.0.0.1:8000
Selanjutnya tinggal buat model di django seperti biasa.
8 Akses ebay SOAP API dengan Python
This is how accessing soap api xml in python.
import json import requests from ebaysdk.response import ResponseDataObject, Response def get_countries(): """Request SOAP API for Country. """ headers = { 'X-EBAY-API-SITEID': 0, 'X-EBAY-API-DEV-NAME': 'your-api-dev', 'X-EBAY-API-CERT-NAME': 'your-api-cert', 'Content-Length': '1096', 'X-EBAY-API-APP-NAME': 'your-api-app', 'X-EBAY-API-COMPATIBILITY-LEVEL': '837', 'X-EBAY-SDK-REQUEST-ID': 'your-sdk-request', 'X-EBAY-API-CALL-NAME': u'GetEbayDetails', 'User-Agent': 'eBaySDK/2.1.2 Python/2.7.6 Linux/3.13.0-37-generic', 'Content-Type': 'text/xml' } body = """ <?xml version="1.0" encoding="utf-8"?> <GeteBayDetailsRequest xmlns="urn:ebay:apis:eBLBaseComponents"> <RequesterCredentials> <eBayAuthToken>your-token</eBayAuthToken> </RequesterCredentials> <DetailName>CountryDetails</DetailName> </GeteBayDetailsRequest> """ resp = requests.post('https://api.ebay.com/ws/api.dll', data=body, headers=headers) from ebaysdk.response import ResponseDataObject, Response xml = resp.content o = ResponseDataObject({'content': xml}, []) r = Response(o, verb='findItemsByProduct', list_nodes=['searchResult.item']) data = json.loads(r.json(), 'utf8') countries = [(i['Country'], i['Description']) for i in data['GeteBayDetailsResponse']['CountryDetails']] return countries
9 Konfigurasi dan Implementasi PostgreSQL HStoreField di django
Install postgresql-contrib-9.3:
sudo apt-get install postgresql-contrib-9.3
Kemudian jalankan postgre console
sudo -u postgres -i psql \connect dbname create extension hstore;
Output:
CREATE EXTENSION
Kemudian deskripsikan model di django:
from django.contrib.postgres.fields import HStoreField class Item(models.Model): itemId = models.CharField(max_length=50) title = models.CharField(max_length=1000) sellerInfo = HStoreField() def __unicode__(self): return self.title
Contoh query:
i = Item(name='kholidfu', data={'skill': 'python'}) i.save()
10 Setting nginx + uwsgi + supervisor + django
Berikut setting nginx:
server { listen 80; server_name www.example.com; location / { try_files $uri @app; } location @app { include uwsgi_params; uwsgi_pass unix:/tmp/uwsgi.sock; } }
Setting uwsgi di supervisord.conf
:
[supervisord] logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log) logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB) logfile_backups=10 ; (num of main logfile rotation backups;default 10) loglevel=info ; (log level;default info; others: debug,warn,trace) pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid) nodaemon=false ; (start in foreground if true;default false) minfds=1024 ; (min. avail startup file descriptors;default 1024) minprocs=200 [program:kresna] command=/home/sopier/kresna2/bin/uwsgi --socket /tmp/uwsgi.sock --module=config.wsgi:application --env DJANGO_SETTINGS_MODULE=config.settings.production --callable app -H /home/sopier/kresna2/ --chdir /home/sopier/kresna2/ --chmod-socket=666 --processes=5 directory=/home/sopier/kresna2/ autostart=true autorestart=true stdout_logfile=/tmp/kresna2.log redirect_stderr=true
Yang harus Anda perhatikan disini adalah letak dari module wsgi
,
karena saya di sini menggunakan cookiecutter dari pydanny, maka secara
default, file wsgi.py
berada di direktori config
. Kalau
menggunakan versi default startproject
dari django, file wsgi.py
berada di direktori myproject.wsgi
.
Selamat mencoba!
11 Port di django vagrant
Jika menggunakan vagrant, setelah VagrantFile terkonfigurasi dengan baik dan benar, agar dapat diakses oleh OS Host, jalankan perintah berikut:
./manage.py runserver [::]:8000
Jangan lupa, sesuaikan port number dengan VagrantFile Anda!