This project is winner of Dataweek hackathon in 2012. PollWiser produces social context for poll results.
There are several steps involved before PollWiser can be used.
-
crawl twitter history using twitter_crawler/crawler.py
Currently, the crawler is not fully autonomous, due to the fact we're using the free search API and not firehose. You need to modify the two lines setting search api url and change the dates to the target date + 1 (i.e. to crawl tweets from Sept 26, you need to change the date to 2012-09-27) And to crawl another day, you have to modify the date in the url every time, the crawler works incrementally.
-
the crawler will generate result.shelve file, copy this file to Sentiment directory and rename it as source.result.
-
generate sentiment result for crawled data by running Sentiment/sentiment.py
-
sentiment.py will generate a file called sentiment_result.shelve
-
run parse_result.py, this will generate a file called parsed.shelve based on source.shelve and sentiment_result.shelve
-
copy parsed.shelve to FirebaseUploader directory and rename it to parsed.data
-
go to firebase data store in browser and delete tweeter_data_2
-
run FirebaseParsedTweetUploader.py to upload parsed.data
TODO: add steps for preparing poll data