Lenin Analysis

Setup

:~/leninanalysis$ [sudo] pip install -r requirements.txt

(Install pip: [sudo] apt-get install python-pip)

Edit config.py. Change data_dir to point to the absolute path of your data directory. This is where all the scrapped JSONs will be saved in.

Also, you can set MIN_YEAR and MAX_YEAR to scrap smaller subsets of Lenin Full Works.

For instance:

data_dir="/home/marat/Dropbox/Facu/incc/incc-2012/TP Lenin/data/"
MIN_YEAR = 1900
MAX_YEAR = 1905
WNA_VERBOSE = 1

$ python cccp.py --scrap

This command creates a file named lenin_work.json at the root directory of the project.

You can also scrap a smaller subset with:

$ python cccp.py --scrap

$ ./cccp.py --populate-database

Run Information Value analysis over the documents (creates collection lenin.information_value_result)

$ ./cccp.py --calculate-results

##Links

Name		Name	Last commit message	Last commit date
Latest commit History 351 Commits
analyzers		analyzers
clustering		clustering
commands		commands
includes		includes
information_value		information_value
lenin		lenin
moby_dick		moby_dick
notebooks		notebooks
paper		paper
plot		plot
test		test
.gitignore		.gitignore
.travis.yml		.travis.yml
README.md		README.md
State and Revolution.ipynb		State and Revolution.ipynb
WNA Test.ipynb		WNA Test.ipynb
analyzers.ipynb		analyzers.ipynb
average_document_analysis.ipynb		average_document_analysis.ipynb
cccp.py		cccp.py
config.py.base		config.py.base
document_analysis.ipynb		document_analysis.ipynb
documents_plot.ipynb		documents_plot.ipynb
experiments.py		experiments.py
interpreter.py		interpreter.py
moby_dick.py		moby_dick.py
requirements.txt		requirements.txt
results.txt		results.txt
scrapy.cfg		scrapy.cfg
synsets.ipynb		synsets.ipynb
top_senses.ipynb		top_senses.ipynb