Political contributions mini search engine

A simple site to make political contribution data from the 総務省 more searchable.

Requirements

To run the site:

To do scraping:

To make thumbnails:

For running the OCR process:

This repo contains no data. To fetch the data you need to:

Scrape PDFs from the 総務省 using tools/scrape.py
Initialise the DB with ./main.py initdb
Generate thumbnails for the PDFs with tools/make_thumbnails.py - this can take a long time as it runs optipng on each doc - use -n to go faster and make slightly larger PNGs.
Import the documents into the database with tools/importdocs.py - ideally run this 3 times: 1. groups only (-g), 2. defer enabled (no options), 3. no-defer (-n).

Each tool has additional options that can be applied. Run with --help to learn more.

The code in this repository is made available under the BSD license. See LICENSE.

Name		Name	Last commit message	Last commit date
Latest commit History 98 Commits
cache		cache
db		db
pdf		pdf
shikin		shikin
static		static
templates		templates
tools		tools
translations/ja/LC_MESSAGES		translations/ja/LC_MESSAGES
v2/toumeika		v2/toumeika
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
Vagrantfile		Vagrantfile
app.wsgi		app.wsgi
babel.cfg		babel.cfg
babel_update.sh		babel_update.sh
bootstrap.sh		bootstrap.sh
deploy.sh		deploy.sh
main.py		main.py
requirements.txt		requirements.txt
test_shikin.py		test_shikin.py
vm_bootstrap.sh		vm_bootstrap.sh