Using tweepy to collect tweets, and google cloud's NLP to analyse the sentiments

This project started as a playfull way to understand the process of interacting with Apis, collect data, clean it, use natural language process to label it, then plot in a way that we can have some conclusions about the data. It endend as a program that automatic collect, clean and language process the data.

Linux Ubuntu 18.04:

Clone this repository in your machine;

Install miniconda3 latest version;

Install mongodb;

Create a conda's virtual enviroment in your pc;

Install the dependencies;

Create an account on google cloud and twitter's development platform;

At twitter's development, activate the apikeys, and copy paste on /crendetials;

At google cloud, activate the cloud natural language API and create a service account, after that, download the .json and paste it on /credentials;

Run in your console python3 index.py ;

Choose the robot you want to use;

When the robot is finished run python3 index.py again and select other robot.

PS: Not tested on Windows and OS X but it may work. The possible problem could be the os.env at the analyser robot.

You can use this code to make a sentiment analyses towards any new media, a new game, new movie, new series, new product.

I called robot, each part of the program:

-Manager coordenates the calls;

-Listener is responsable for collecting the tweets;

-Saver is responsable for interacting with mongodb;

-Preprocessor is responsable for cleaning the tweets;

-Analyser is responsable for request google's api;

-Posprocessor is responsable to deliver the final csv.

Feel free to suggest improvements, especially at the preprocessor, cause you will not get only the tweets you want.

In this especific case I used Joker's new movie as an use case.

After the final csv is complete, you can use any tool to plot your data. I choose datastudio from google, cause I found it easy and intuitive to use.

See how beautiful it became =]

You can interact with the plots here.

After setting up your enviroment, install the dependencies running:

conda install $(cat dependencies/conda_dep.txt)

then:

pip install $(cat dependencies/pip_dep.txt)

Feel free to ask me if you have any doubts or problems with the program =]

Thomas Roquetti – @ThomasBorgess – thomas.mr.borges@gmail.com

Distributed under the MIT license. See LICENSE for more information.

https://github.com/ThomasBorges

Please, be free to contribute, I'll appreciate it =]

1. Fork it (https://github.com/ThomasBorges/joker-project/fork)
2. Create your feature branch (git checkout -b feature/fooBar)
3. Commit your changes (git commit -am 'Add some fooBar')
4. Push to the branch (git push origin feature/fooBar)
5. Create a new Pull Request