Amateur experiments in Data Analysis and VISualizations.

Actually an excuse to learn a thing or two about data analysis and to play around with Pandas and d3js.

You will need Python and few other dependencies for running the examples. The best way is to create a new virtualenv and install all python based dependencies :

$ virtualenv davisenv --no-site-packages
$ source davisenv/bin/activate
(davisenv)$ cd /path/to/davis/repo
(davisenv)$ pip install -r requirements.txt

To be able to run the examples, you will need to download and prepare some data first. For now, just download a day's activity from githubarchive.org as follows:

$ cd githubarchives
$ wget http://data.githubarchive.org/2012-11-05-{0..23}.json.gz

Next, convert the json files into a single csv files as follows:

(davisenv)$ cd ../
(davisenv)$ python
>>> import glob
>>> from githubarchive import csvify_activities
>>> files = glob.glob('githubarchives/2012-11-05-*.json.gz')
>>> csvify_activities(files, 'githubarchives/activities-2012-11-05.csv')

All examples are packaged as a webapp using the Flask web framework and therefore can be run the browser by running the development server locally. Assuming that Flask would have already been installed from the requirements.txt file, you can start the dev server as follows:

(davisenv)$ python app.py

Now open http://127.0.0.1:5000 in your browser.