A harvester app for twitter data
Make sure you have this stack installed on your machine first
To avoid any conflict with system dependencies, run the installation under virtual environment.
$ pip install -r rquirements.txt
Copy data/config.json.example and rename it to data/config.json. Adjust the value in it to your own system configuration.
The script contain two main components
- Producer Fetch twitter data from streaming API and push it to message queue
- Consumer Consume twitter data from RabbitMQ queue and store it to CouchDB collection
$ make producer
$ make consumer
- Dockerize the app
- Run the whole system with
docker-compose