Skip to content

cormac/open-corroborator

 
 

Repository files navigation

corroborator

Case management with data validation, corroboration, omission and duplication checking

Installation

virtualenv should be used for package management To install virtualenv

[sudo] apt-get install python-virtualenv

Other packages are required for modules and other build dependencies:

[sudo] apt-get install libmysqlclient-dev python-dev mysql-client libxslt-dev libgeos-c1 ffmpeg git libjpeg-dev

Install local python environment

From the root folder run:

virtualenv --python=python2.7 env --no-site-packages

Install required packages

env/bin/easy_install -U distribute
env/bin/pip install -r requirements.txt

Set up system celeryd install

Install celeryd

apt-get install python-celery

On Debian, the configuration file for celeryd exists in /etc/default/celeryd - an example configuration file is provided in conf/celeryd.conf. Paths in this configuration file must be modified first. The celery daemon must be restarted once this file is installed via /etc/init.d/celeryd restart.

pull in git submodules

git submodule init
git submodule update --recursive

Create and configure database

In mysql, create a database and user for the app:

CREATE DATABASE corroborator_staging;
CREATE USER 'django'@'localhost' identified by 'password';
GRANT ALL ON corroborator_staging.* to 'django'@'localhost';

Add this username and password combination to DATABASE in corroborator/settings/staging.py.

Initial db sync

If you have no database set up, run

env/bin/python2.7 managestaging.py syncdb
env/bin/python2.7 managestaging.py migrate

Solr installation

Solr needs a few packages installed before use:

[sudo] apt-get install supervisor openjdk-6-jdk

Solr installation is a simple matter of untarring and chowning the file as is appropriate for your webserver configuration. Some extra configuration is required, as detailed in the section below.

Solr can be tempermental so we run it via Supervisord. An example configuration file for solr exists in conf/solr.supervisor.conf

  • it can be dropped into /etc/supervisor/conf.d/ on Debian. Make sure to change the paths as required.

Other configuration

Assuming that you have Solr and Apache all good to go, you should be ready now.

If you encounter the error ApiKey matching query does not exist. Lookup parameters were {'user': <User: root>} (or for another user), create an API key in the https://yoursite/admin panel.

Solr Usage

Changes to model schema where new fields are added or removed from Bulletin, Incident or Actor models will require an update to the solr schema. To do this the schema.xml file must be updated. Haystack will generate a new schema.xml file with the command

env/bin/python manage.py build_solr_schema > schema.xml --settings=[local settings]

from the root dir of the project

you must then copy this file to the conf folder of your solr instance and add this line under the fields section

<field name="_version_" type="long" indexed="true" stored="true"/>

The schema can then be re-generated by running

env/bin/python manage.py rebuild_index --settings=[local settings]

About

Data analysis and fact corroboration

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 49.7%
  • JavaScript 33.3%
  • CSS 10.2%
  • Smarty 4.5%
  • HTML 2.2%
  • Ruby 0.1%