Code for making an API to contribute data to DC2's data lake

Running the Flask API Locally:

REQUIRED SERVICES

To run locally, you will need to have a redis server and amqp server running and a celery worker with the adjunct to accept jobs.

On ubuntu 12.04 or later, simply run to install these services and begin running them on default ports.

sudo apt-get redis-server python-celery

DEPENDENCIES

Python packages required include:

flask boto redis amqp magic xlrd

Install with:

sudo easy_install magic sudo pip install Flask boto redis amqp xlrd

Note that several of these dependencies overlap with the adjunct processor.

Additionally, you will need the ThreeScalePY package which must be downloaded from github and installed manually. Clone this repository and follow instructions for setup: https://github.com/3scale/3scale_ws_api_for_python

CONFIGURATION

Copy sample_config.py to config.py and alter the appropriate variables if necessary.

RUNNING

To run the api, type:

python api.py

This terminal will now display status messages from the API.

CALLING

The api will be running on your localhost with port 5000 by default.

To call, use the following example curl calls with valid app_id and app_key pairs:

POST file data: curl -i -F "app_id=12345EXAMPLE" -F "app_key=abc6943jqq509x26EXAMPLE" -F "file=@/path/to/1984_Excerpt.txt;type=text/plain" "http://127.0.0.1:5000/v1/documents"

GET the status of a job: curl -i -X GET "http://127.0.0.1:5000/v1/documents/queue/f4d732ae-4264-40b5-9540-bf75f4799a76?app_id=12345EXAMPLE&app_key=abc6943jqq509x26EXAMPLE"

GET the location of the result files: curl -i -X GET "http://127.0.0.1:5000/v1/documents/f4d732ae-4264-40b5-9540-bf75f4799a76?app_id=12345EXAMPLE&app_key=abc6943jqq509x26EXAMPLE"

MONITORING

You can monitor the status of the API from either the terminal from which it was started OR by viewing the api.log file.

CHANGELOG