• Install and use https://github.com/mbi/django-rosetta for sane translation management
• Setup auto-restart of Nginx and Gunicorn in case server goes down
• Setup daily auto backup of MySQL. Look at moving DB to dedicated DB servers.
• Tag 4.0.0 release
• About Secretariat page should have links to profiles
• Finish FAQ update and send to translation
• File upload field for forum comments
• Make own gravatar_url tag to use instead of Mezzanine's in order to be able to serve over SSL
• User registration open but behind login-required and staff-user required. The URL to add new users is /account/signup/
• Setup Versioning of all page content
• Remove the unused Blog app from admin
• Eventually update to Mezzanine 3.1.5
  • Update to latest version of Mezzanine and make sure current functionality works
• Document nginx/gunicorn/supervisor setup (currently running gunicorn with deprecated gunicorn_django -b 0.0.0.0:8000 command — get it running and working with recommended command instead)
• Last modified date for pages
• IRSS refactor
  • The easiest way to implement this is probably to use CAS-Provider and CAS-consumer or django-cas. Another option: mama-cas. Relevant documentation pages: multiple databases, authentication, multiple sites framework. See also this blog post.
• Phytosanitary.info refactor (use original code?)
• Both phytosantiary.info and apppc.org will need to get SSL certificates for single sign-on to work securely: > Even if the authentication with CAS is made using a mechanism which makes it difficult to interfere with, all authorized communication will subsequentely use a cookie identifing the session which can be used to hijack the connection. So you need to encrypt the communication. There is just no way around that if you want to enforce some sort of security.
• Order permission groups alphabetically in admin
• Setup working fabric script for easy deployment that does the following after running fab deploy dev:
  1. Adds, commits and pushes files to Github (for future: if tests pass)
  2. Logs in to dev.ippc.int, activates application virtualenv and pulls changes from Github
  3. Collect static files to locations to be served on dev server
  4. Restart gunicorn and nginx
• If no publication or agenda numbers exist, don't show header or cells
• wiki.ippc.int

1. Install Django development environment on your computer and follow instructions to get it running: http://wiki.bitnami.com/Infrastructure_Stacks/BitNami_Django_Stack

2. Clone code repository (currently on GitHub), move into it and install third-party libraries for the project:

   git clone https://github.com/hypertexthero/ippcdj.git ippcdj_repo
   cd ippcdj_repo
   pip2.7 install -r requirements/project.txt
   mv local_settings_example.py local_settings.py 
   python manage.py createdb
   python manage.py runserver
   
   

3. Go to 127.0.0.1:8000 to see the app running. go to 127.0.0.1:8000/admin to log in to the admin interface. To stop the server press Ctrl-C (Ctrl-Z and Enter in Windows) in the terminal.

GitHub accounts are for codebase repository. Copies of codebase repository are also available in:

1. Each developer's computer
2. dev.ippc.int server
3. www.ippc.int server

We're using a hared repository model and the main IPPC code is at

First, setup git (so you don't have to keep putting in your password every time).

Then, here's a basic guide (below is an example of working with this repository itself).

1. If you're working on the code for the first time, first clone repository from ippc repo (the first time you start working with it)

   cd ~/projects
   git clone https://github.com/ippc/ippcdj.git

2. Before beginning the day's work, pull the latest changes from the online repo

   git pull
   

3. Change to ippcdj_repo directory and open the directory with your text editor (in this case, mate == textmate editor)

   cd ippcdj_repo && mate .

4. At end of day or more often as you prefer, add your changes to your local staging area and commit them to your local repo

   git add .
   git commit -m 'new work'

5. Push your changes to main server at end of day (or more often)

   git push origin master

1. Create a new branch called 'newfeature' and switch to it

   git checkout -b newfeature

2. Create a new file for this new feature and add some text to it

   echo 'a text file!' >> newfile.txt

3. At end of day or more often as you prefer, commit the changes to this branch

   git add .
   git commit -m 'new feature'

4. Push your branch to main repo at end of day (or more often)

   git push origin newfeature

• Understanding the GitHub Flow (brief visual overview)
• GitHub Flow in the Browser (In-browser GitHub features)
• GitHub Flow (detailed overview)
• Pull new updates from original Github repository into forked Github repository
• Git for beginners

IPPC Repository: https://github.com/hypertexthero/ippcdj - The master branch that should eventually be the same code that is live at production website. Another, likely better, option is to create an IPPC Organization page and move this there.

https://docs.djangoproject.com/en/dev/topics/i18n/translation/

Edit the django.po file for each language in ippcdj_repo/conf/locale/ and then run the following commands in the terminal to compile the translation files:

python manage.py makemessages --all  
python manage.py compilemessages

Dev server exlqaippc2.ext.fao.org setup and configuration for IPPC 4.0 prototype at http://dev.ippc.int/en/ (only available within FAO network). To update code (eventually this will all be done with one command which fires a fabric script such as fab deploy dev):

1. ssh root@hqldvippc2.hq.un.fao.org

    ssh root@hqldvippc2.hq.un.fao.org
   

2. Change directory to the project and activate virtualenv

    cd /work/projects/ippcdj-env && . bin/activate
   

3. Change to repository directory

    cd	ippcdj_repo/
   

4. Pull latest changes. If you get a warning about overwriting existing changes, do git stash save --keep-index then git stash drop http://stackoverflow.com/a/14318266

    git pull
   

5. move any static media to proper serving location python manage.py collectstatic Not necessary anymore as we have configured nginx to look in the right places for static media.

6. Run any data migrations on the database: Right now we're just transfering the whole dump from dev to the MySQL server when models are updated.

    python manage.py schemamigration ippc --auto
    python  manage.py migrate ippc
   

7. Compile and make translations

    python manage.py makemessages --all  
    python manage.py compilemessages
   

   If some translations, especially ones in blocktrans don't show up, check http://stackoverflow.com/questions/1377372/django-fuzzy-string-translation-not-showing-up

8. Stop and restart Gunicorn application server:

   This command shows you the master PID of Gunicorn if it is running (http://stackoverflow.com/a/26926130):

    pstree -ap|grep gunicorn
   

   Restart Gunicorn gracefully with the following (replace with the process number, for example, kill -HUP 10745):

    kill -HUP <pid>   
   

   If the above doesn't work and it is an emergency, just do the following (NOTE: THE SITE WILL GO DOWN):

    pkill gunicorn  
    gunicorn_django -b 0.0.0.0:8000 --daemon --log-file /var/log/nginx/gunicorn-beta-ippc-error.log  
   

   Note: If you are starting Gunicorn for the first time, the command is the last one above beginning with gunicorn_django

9. Restart Nginx reverse-proxy server (web-facing) server

    service nginx restart
   

	# http://stackoverflow.com/a/24820722
	map $http_user_agent $limit_bots {
	     default 0;
	     ~*(google|bing|yandex|msnbot) 1;
	     ~*(AltaVista|Googlebot|Slurp|BlackWidow|Bot|ChinaClaw|Custo|DISCo|Download|Demon|eCatch|EirGrabber|EmailSiphon|EmailWolf|SuperHTTP|Surfbot|WebWhacker) 1;
	     ~*(Express|WebPictures|ExtractorPro|EyeNetIE|FlashGet|GetRight|GetWeb!|Go!Zilla|Go-Ahead-Got-It|GrabNet|Grafula|HMView|Go!Zilla|Go-Ahead-Got-It) 1;
	     ~*(rafula|HMView|HTTrack|Stripper|Sucker|Indy|InterGET|Ninja|JetCar|Spider|larbin|LeechFTP|Downloader|tool|Navroad|NearSite|NetAnts|tAkeOut|WWWOFFLE) 1;
	     ~*(GrabNet|NetSpider|Vampire|NetZIP|Octopus|Offline|PageGrabber|Foto|pavuk|pcBrowser|RealDownload|ReGet|SiteSnagger|SmartDownload|SuperBot|WebSpider) 1;
	     ~*(Teleport|VoidEYE|Collector|WebAuto|WebCopier|WebFetch|WebGo|WebLeacher|WebReaper|WebSauger|eXtractor|Quester|WebStripper|WebZIP|Wget|Widow|Zeus) 1;
	     ~*(Twengabot|htmlparser|libwww|Python|perl|urllib|scan|Curl|email|PycURL|Pyth|PyQ|WebCollector|WebCopy|webcraw) 1;
	 } 
	 
	# http://stackoverflow.com/a/18338015
	ssl_certificate      /etc/pki/tls/certs/name.of.your.cert.chained.crt;
	ssl_certificate_key  /etc/pki/tls/private/name_of_your_key.key;
	ssl_session_timeout  5m;
	ssl_protocols  SSLv3 TLSv1;
	ssl_ciphers  ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+EXP;
	ssl_prefer_server_ciphers   on;
	
	index index.php index.htm index.html;
	
	## www.domain.tld ####################
	
	# redirect http://domain.tld and https://domain.tld to https://www.domain.tld
	server {
	       listen xxx.xx.xx.xxx:80;
				 listen xxx.xx.xx.xxx:443 ssl;
				 server_name domain.tld;
	       return 301 https://www.domain.tld$request_uri;
	}
	
	# redirect http://www.domain.tld to https://www.domain.tld
	server {
	       listen xxx.xx.xx.xxx:80;
				 server_name www.domain.tld;
	       return 301 https://$server_name$request_uri;
	}
	
	server {
		listen xxx.xx.xx.xxx:443 ssl;
		server_name www.domain.tld;
		root /work/projects/ippcdj-env/public;
		
		location /admin/media/ {
			alias /work/projects/ippcdj-env/lib/python2.7/site-packages/django/contrib;
		}
		
		location /static/media {
			alias /work/projects/ippcdj-env/public/static/media;
		}
		
		location /static {
			alias /work/projects/ippcdj-env/ippcdj_repo/static;
		}
		
		location /largefiles {
			alias /work/projects/ippcdj-env/public/largefiles;
		}
		
		location /robots.txt {
			alias /work/projects/ippcdj-env/ippcdj_repo/static/robots.txt;
		}
		
		location / {
			proxy_set_header X-Real-IP  $remote_addr;
			proxy_set_header X-Forwarded-For $remote_addr;
			proxy_set_header Host $host;
			proxy_pass http://127.0.0.1:8000;
		}

		# Deny illegal Host headers
		# http://stackoverflow.com/a/17477436
		 if ($host !~* ^(domain.tld|www.domain.tld)$ ) {
		   return 444;
		 }
		 
		# Deny annoying bots (see mapping at top of file)
		# http://stackoverflow.com/a/24820722
		if ($limit_bots = 1) {
		 	return 403;
		}
		
		# what to serve if upstream is not available or crashes
		error_page 500 502 503 504 /var/www/html/index.html;
		
		# don't display .htaccess files
		location ~ /\.ht {
			deny all;
		}	
	}
	
	## html.staticsitedomain.tld ####################
	server {
		listen xxx.xx.xx.xxx:80;
		server_name html.staticsitedomain.tld;
		root /var/www/html/html.staticsitedomain.tld;
		index index.php index.htm index.html;

		location / {
			try_files $uri $uri/ /index.php?q=$uri&$args;
		}
		
		location ~ /\.ht {
			deny all;
		}
	}
	
	## www.phpsitedomain.tld ####################
	server {
		listen xxx.xx.xx.xxx:80;
		server_name phpsitedomain.tld www.phpsitedomain.tld;
		root /opt/lampp/htdocs/prodsite;
		
		location / {
			try_files $uri $uri/ /index.php?q=$uri&$args;
	  }
		
	  location ~ \.php$ {
	    expires    off;
	    include /etc/nginx/fastcgi_params;
	    fastcgi_pass    127.0.0.1:9000;
	    fastcgi_index   index.php;
	    fastcgi_param   SCRIPT_FILENAME  $document_root$fastcgi_script_name;
	  }
		
		location ~ /\.ht {
			deny all;
		}
		error_page 500 502 503 504 /var/www/html/index.html;
	}

MySQL, php and phpmyadmin

Start:

/etc/init.d/mysql start

• Implemented using django-guardian

Request ----> Reverse-Proxy Server (Nginx)
                 |
                  \                           
                   `-> App Server (Gunicorn). 127.0.0.1:8081 --> Django app

php-fpm

The Django site runs behing a Nginx reverse-proxy server which routes dynamic (non-static requests such as img, pdf, css, js, etc) requests to Gunicorn application server, which routes them to the Django app. Other Drupal/PHP sites on the same server receiving requests from Nginx use php-fpm, which needs to be running:

service php-fpm status
service php-fpm start
# Sometimes php-fpm might have broken instances running, preventing a restart. This command is a clean way to clear them out and restart php-fpm
killall -9 php-fpm; service php-fpm restart

Current Software Stack

• Python 2.7.6, Pip and Virtualenv[^1]
• Nginx web & reverse proxy server
• Gunicorn web application WSGI server

The main web application is built with Django and Mezzanine.