Osm Export Tool platform allows you to create custom OpenStreetMap exports for various HOT regions. You can specify an area of interest and a list of features (OpenStreetMap tags) for the export. A current OpenStreetMap data extract for that area in various data formats is then created for you within minutes.

The live site http://export.hotosm.org is currently still powered by the older version 1 code living at https://github.com/hotosm/hot-exports

This repo contains the newly re-written version 2 of the OSM exports tool.

Some prior experience with Django would be helpful, but not strictly necessary.

$ sudo apt-get update
$ sudo apt-get upgrade

HOT Exports requires Python 2.7.x.

To install pip, run:

$ sudo apt-get install python-pip

Git is used for version control. To check your version, and if you have it installed:

$ git --version or run $ sudo apt-get install git

Virtualenv (virtual environment) creates self-contained environments to prevent different versions of python libraries/packages from conflicting with each other.

To make your life easier install virtualenvwrapper

$sudo pip install virtualenvwrapper

Add the following to .bashrc or .profile

export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/dev
source /usr/local/bin/virtualenvwrapper.sh

Run source ~/.bashrc

Run mkvirtualenv hotosm to create the hotosm virtual environment.

Change to the $HOME/dev/hotosm directory and run workon hotosm.

Install PostgreSQL / PostGIS and its dependencies,

$ sudo apt-get install libpq-dev python-dev

$ sudo apt-get install postgresql postgresql-contrib

$ sudo apt-get install postgis postgresql-9.3-postgis-2.1

$ sudo su - postgres
$ createdb 'hot_exports_dev'
$ create role hot with password '<-password->'

You might need to update the pg_hba.conf file to allow localhost connections via tcp/ip or allow trusted connections from localhost.

Run $ psql -h localhost -U hot -W hot_exports_dev

$ ALTER ROLE hot SUPERUSER;
$ ALTER ROLE hot WITH LOGIN;
$ GRANT ALL PRIVILEGES ON DATABASE hot_exports_dev TO hot;
$ CREATE EXTENSION POSTGIS;
$ CREATE EXTENSION HSTORE;

Create the exports schema

$ CREATE SCHEMA exports AUTHORIZATION hot;

We need gdal >=1.10.0

$ sudo apt-get install software-properties-common
$ sudo add-apt-repository ppa:ubuntugis/ubuntugis-unstable
$ sudo apt-get update
$ sudo apt-get install gdal-bin libgdal-dev

To install the python GDAL bindings into your virtualenv you need to tell pip where to find the libgdal header files, so in your shell run:

$ export CPLUS_INCLUDE_PATH=/usr/include/gdal
$ export C_INCLUDE_PATH=/usr/include/gdal

The HOT Export pipeline depends on a number of third-party tools.

$ sudo apt-get install osmctools

$ sudo apt-get install spatialite-bin libspatialite5 libspatialite-dev

$ sudo apt-get install default-jre zip unzip

Download the latest version of the mkgmap utility for making garmin IMG files from http://www.mkgmap.org.uk/download/mkgmap.html

Download the latest version of the splitter utility for splitting larger osm files into tiles. http://www.mkgmap.org.uk/download/splitter.html

Create a directory and unpack the mkgmap and splitter archives into it.

For details on the OSMAnd Map Creator utility see http://wiki.openstreetmap.org/wiki/OsmAndMapCreator

Download the OSMAnd MapCreator from http://download.osmand.net/latest-night-build/OsmAndMapCreator-main.zip. Unpack this into a directory somewhere.

HOT Exports depends on the rabbitmq-server. For more detailed installation instructions see http://www.rabbitmq.com/install-debian.html. The default configuration should be fine for development purposes.

$ sudo apt-get install rabbitmq-server

In the hotosm project directory run:

$ git clone git@github.com:hotosm/osm-export-tool2.git

From the project directory, install the dependencies into your virtualenv:

$ pip install -r requirements-dev.txt

or

$ pip install -r requirements.txt

Create a settings_private.py file in the hot_exports directory and add the following:

DATABASES = {
    'default': {
        'ENGINE': 'django.contrib.gis.db.backends.postgis',
        'NAME': 'hot_exports_dev',
        'OPTIONS': {
            'options': '-c search_path=exports,public'
        },
        'CONN_MAX_AGE': None,
        'HOST': 'localhost',
        'USER': 'hot',
        'PASSWORD': 'your password',
    }
}

Update the settings.py file. Set the following config variables:

Set OSMAND_MAP_CREATOR_DIR to the directory where you installed the OSMAnd MapCreator.

Set GARMIN_CONFIG to point to the absolute path to the garmin configuration file, ./utils/conf/garmin_config.xml by default.

Update the utils/conf/garmin_config.xml file. Update the garmin and splitter elements to point to the absolute location of the mkgmap.jar and splitter.jar utilites.

Once you've got all the dependencies installed, run ./manage.py migrate to set up the database tables etc.. Then run ./manage.py runserver to run the server. You should then be able to browse to http://localhost:8000/

HOT Exports depends on the Celery distributed task queue. As export jobs are created they are pushed to a Celery Worker for processing. At least two celery workers need to be started as follows:

From a 'hotosm' virtualenv directory (use screen), run:

$ celery -A hot_exports worker --loglevel debug --logfile=celery.log.

This will start a celery worker which will process export tasks. An additional celery worker needs to be started to handle purging of expired unpublished export jobs. From another hotosm virtualenv terminal session, run:

$ celery -A hot_exports beat --loglevel debug --logfile=celery-beat.log

See the CELERYBEAT_SCHEDULE setting in settings.py.

For more detailed information on Celery Workers see here

HOT Exports uses bower to manage javascript dependencies.

sudo apt-get install nodejs
sudo apt-get install npm
sudo npm -g install bower
sudo npm -g install yuglify

To work with Transifex you need to create ~/.transifexrc, and modify it's access privileges chmod 600 ~/.transifexrc

Example .transifexrc file:

[https://www.transifex.com]
hostname = https://www.transifex.com
password = my_super_password
token =
username = my_transifex_username

• update source language (English) file

  • python manage.py makemessages -l en

• push the new source file to Transifex service, it will overwrite current source file

  • tx push -s

• when adding a new language, it's resource file does not exists in the project, but it's ok as it will be automatically created when pulling new translations from the service
  • add a local mapping: tx set -r osm-export-tool2.master -l hr osmtm/locale/hr/LC_MESSAGES/osmtm.po
• after there are some translation updates, pull latest changes for mapped resources
  • for a specific language(s):
    • tx pull -l fr,hr
  • for all languages:
    • tx pull
• finally, compile language files
  • python manage.py compilemessages