Skip to content

Fractal-Software/fractalDataStream

Repository files navigation

Sentiment Analysis Pipeline

Container Memory Size must be set to atleast 6G, for Ubuntu see:

https://hostadvice.com/how-to/how-to-limit-a-docker-containers-resources-on-ubuntu-18-04/

For mac and Windows this can be set on Docker > Preferences

This application was made for the purpose of creating a sentiment analysis pipeline that connects to twitter -for a given subject-, and sends the processed data to the backend.

Start the application:

Initialize docker swarm

docker swarm init

The external secrets must be created before you deploy the docker-compose.yml file:

$ echo "secret content" | docker secret create consumer_key -
$ echo "secret content" | docker secret create consumer_secret -
$ echo "secret content" | docker secret create access_key -
$ echo "secret content" | docker secret create access_secret -

Build the image:

$ docker-compose build

Deploy the stack with docker stack:

$ docker stack deploy -c docker-compose.yml fractal-pipeline

Deploy Postgres Sink Connector:

$ curl -X POST -H "Accept:application/json" -H "Content-Type: application/json" --data @postgres-sink.json http://localhost:8083/connectors