Mturk-Tracker is a Django web application and is setup within python virtualenv and requires a working python envioroment and PostgreSQL database.

Mturk-Tracker requires base packages:

$ sudo apt-get install python-virtualenv subversion mercurial \
        postgresql-8.4 postgresql-server-dev-8.4  python2.6-dev \
        libevent-dev

To initialize project, virtualenv python package is required (you may also want to use virtualenvwrapper extension). First, create and activate new virtual python environment:

$ virtualenv mturk --no-site-packages
$ cd  mturk
$ . bin/activate

or:

$ mkvirtualenv mturk --no-site-packages
$ workon mturk
$ cd $VIRTUAL_ENV

if using virtualenvwrapper.

After that, clone mturk code from repository and install all dependencies using pip (you have to install mercurial and subversion first):

$ git clone git://github.com/mickek/Mturk-Tracker.git src
$ cd src
$ git fetch
$ git checkout -b virtualenv --track origin/virtualenv
$ echo "mturk.settings.base" > DJANGO_SETTINGS_MODULE
$ pip install -r requirements.txt

Because pip should take care of all libraries, use it to update already existing configuration. Whenever new dependency appears, run pip -r requirements.txt just to update.

By default mturk.settings.defaults configuration module is being used. To add custom variables you can add code to:

• mturk.settings.default - project default variables visible for all other configuration files

You can also setup any other configuration module by setting DJANGO_SETTINGS_MODULE shell variable or file as given in example above.

Make sure that django app can connect to database, the best way to do that is to allow postgres to accept local connections by editing pg_hba.conf file. Check if you can connect to database:

$ psql -U postgres

In order to setup a clean db you have to create the database and populate it with tables:

$ createdb -U postgres  mturk_tracker
$ createlang plpgsql -U postgres -d mturk_tracker
$ python manage.py syncdb
$ python manage.py migrate

Nothing special, just type:

$ sudo python manage.py runserver

in django project directory. And then point your browser to http://localhost:8000/

You may launch initial crawl by:

$ python manage.py crawl --workers=6 --logconf=logging.conf

Logs will be saved in /tmp/crawler.log. Because mturk requires authentication for HITs listings pagination, use --mturk-email and --mturk-password flags to authenticate and crawl as mturk worker.

To generate data that will be displayed on graphs you need to launch scripts:

$ python manage.py db_refresh_mviews
$ python manage.py db_update_agregates
$ python manage.py db_calculate_daily_stats

TODO