If you want to run this project in a virtualenv to isolate it from other Python projects on your system, create a virtualenv and activate it. Then run bin/install-reqs
to install the dependencies for this project into your Python environment. Python 2.7 is required.
You'll need a PostGIS-enabled PostgreSQL 9.0 database, with the citext
contrib module loaded into it, for this project. See the GeoDjango installation documentation for more details on setting up PostGIS and a PostGIS template database. To enable the citext
module, connect to the template_postgis
database as the Postgres superuser and run \i /usr/share/postgresql/9.0/contrib/citext.sql
(this is the location of the citext.sql
file on Ubuntu; will vary depending on your Postgres install). Once you have a PostGIS template database with citext
enabled, create your database for MLT with a command like createdb -T template_postgis mlt
.
You'll probably need to create an mlt/settings/local.py
file with some details of your local configuration, including, most likely, your database name and user (unless they are both named "mlt", the default). See mlt/settings/local.sample.py
for a sample that can be copied to mlt/settings/local.py
and modified.
Once this configuration is done, you should be able to run ./manage.py syncdb --migrate
, then ./manage.py runserver
and access the MLT in your browser at http://localhost:8000
.
To install the necessary Ruby Gems for Compass/Sass development, run bin/install-gems requirements/gems.txt
. Update requirements/gems.txt
if newer gems should be used.
The MLT includes a user interface for importing batches of addresses from a CSV file, but this interface can't be used for initially populating the MLT, as the number of addresses in the initial import is likely too large to complete within a reasonable time for a web request. In order to manually populate initial addresses, run python manage.py shell
and then the following commands in the shell.
This assumes a four-column CSV file, with a header row, with columns for pl
, street
, city
, state
. A user is necessary so that the addresses appear in the changelog as "created"; swap out the username below for an actual user that you've already created:
>>> from django.contrib.auth.models import User
>>> user = User.objects.get(username="example")
>>> from mlt.map.importer import CSVAddressImporter
>>> importer = CSVAddressImporter(
... user, fieldnames=["pl", "street", "city", "state"], header=True)
>>> importer.process_file("/path/to/addresses.csv")
Note that this will take quite a while depending on the number of addresses; possibly as long as several hours.
In addition to the above configuration, in any production deployment this entire app should be served exclusively over HTTPS (since almost all use of the site is authenticated, and serving authenticated pages over HTTP invites session hijacking attacks). Ideally, the non-HTTP URLs should redirect to the HTTPS version. The SESSION_COOKIE_SECURE
setting should be set to True
when the app is served over HTTPS.
There is an alternate settings file available, mlt/settings/prod.py
which pre-sets some settings that are generally appropriate for a production deployment. Set the DJANGO_SETTINGS_MODULE
environment variable to mlt.settings.prod
in order to use this settings file (overrides in mlt/settings/local.py
will still be respected).
The production settings also require a Celery daemon process, celeryd
, to be running, in order to handle some tasks asynchronously and speed up the user experience (if celeryd
is not running, parcel-loading will not work, and bulk address changes will not be recorded in the change history). For quick testing and debugging, you can simply run python manage.py celeryd -l info
in a terminal. For real deployment, you'll want to daemonize it and run it in the background; see the documentation on running celeryd as a daemon. You may also want to read the Celery monitoring and management guide for more in-depth information about monitoring and managing your Celery instance.
This app also uses the new staticfiles contrib app in Django 1.3 for collecting static assets from reusable components into a single directory for production serving. Under "runserver" in development this is handled automatically. In production, run ./manage.py collectstatic
to collect all static assets into the collected-assets
directory (or whatever STATIC_ROOT
is set to in settings_local.py
), and make those collected assets available by HTTP at the STATIC_URL
setting.
A simple JSON API is available for querying addresses and batches. The API is available at the root URL of /api/v1/
.
Every API call must include the HTTP header X-Api-Key
, whose value must be a valid API key. API keys can be created via the MLT admin interface. An API call without a valid API key will return an HTTP 403 response, with the following body:
{"success": False, "error": "Invalid API key."}
The response to a query to the top-level API URL (/api/v1/
) will have a resource_urls
key listing the resource URLs available via the API. Currently these are:
{
"resource_urls":
{
"addresses": "/api/v1/addresses/",
"batches": "/api/v1/batches/",
},
"success": True,
}
The body of every API response includes a top-level boolean success
key. The value of this key will be true if the request completed successfully; false if an error occurred. In the latter case the body will also include an error
key describing the nature of the error.
List resource responses will also include a total
key, giving the total number of resources matching the given filters (even though not all might be displayed due to paging).
Every resource list URL can accept one or more sort fields via the sort
key in the URL querystring. Any field of the returned data for that resource type can be sorted on; valid fields are listed in the reference for that resource type. Prepend a -
to the field name to sort descending rather than ascending on that field. An example multi-field sorted query URL:
/api/v1/batches/?sort=city&sort=-street
All list resources are paged by default, with a default page size of 20 items. Paging is controlled by offset/limit via start
and num
keys in the URL querystring, rather than by page number. Results will begin with the start
-th item, and num
items will be returned. For example, the following query will return 10 addresses, beginning with the 11th address (in other words, the second page of size-10 pages):
/api/v1/addresses/?start=11&num=10
List resources can be filtered by the value of fields on the resource (see below for full list of fields for each resource type). Filters are provided in the URL querystring:
/api/v1/batches/?tag=foo
Timestamp fields can be filtered on using "[date]" or "[date1] to [date2]", e.g.:
/api/v1/batches/?timestamp=11/5/2011+to+11/10/2011
Each address result includes the following fields:
id
street
city
state
street_number
street_prefix
street_name
street_type
street_suffix
notes
multi_units
complex_name
pl
mapped_by
mapped_timestamp
needs_review
batches
The mapped_by
field should be sorted/filtered as mapped_by__username
, e.g.:
/api/v1/addresses/?mapped_by__username=blametern
The batches
field contains a list of batches the address was imported as part of; each batch will have user
, timestamp
, and tag
keys. Addresses can be filtered by batch using batches__tag
, e.g.:
/api/v1/addresses/?batches__tag=foo
Addresses can be sorted by latest batch timestamp using latest_batch_timestamp
:
/api/v1/addresses/?sort=latest_batch_timestamp
Each batch includes the following fields:
timestamp
tag
user
addresses_url
The addresses_url
field is the API URL to get a list of all addresses in this batch.
The user
field should be sorted or filtered as user__username
.