Scrape the Marion County, Oregon booking log.
First, ensure the following environment variables are set and exported:
POSTGRES_PORTPOSTGRES_DATABASEPOSTGRES_USERPOSTGRES_PASSWORD
Create a Postgres (11+) database. docker can be used to create one:
docker run \
-d \
-p ${POSTGRES_PORT}:5432 \
-e POSTGRES_USER=${POSTGRES_USER} \
-e POSTGRES_DB=${POSTGRES_DATABASE} \
-e POSTGRES_PASSWORD=${POSTGRES_PASSWORD} \
--name=marion-jail-db \
postgres:11Then build the container for the web scraper:
docker build . --tag=marion-jail-scraperUse the web scraper container to setup the database:
docker run \
--rm \
--net=host \
-e POSTGRES_HOST=localhost \
-e POSTGRES_PORT=${POSTGRES_PORT} \
-e POSTGRES_DATABASE=${POSTGRES_DATABASE} \
-e POSTGRES_USER=${POSTGRES_USER} \
-e POSTGRES_PASSWORD=${POSTGRES_PASSWORD} \
--name=marion-jail-scraper \
marion-jail-scraper:latest \
--setup-dbAnd run the scraper, sending logging statements to stdout:
docker run \
--rm \
--net=host \
-e POSTGRES_HOST=localhost \
-e POSTGRES_PORT=${POSTGRES_PORT} \
-e POSTGRES_DATABASE=${POSTGRES_DATABASE} \
-e POSTGRES_USER=${POSTGRES_USER} \
-e POSTGRES_PASSWORD=${POSTGRES_PASSWORD} \
--name=marion-jail-scraper \
marion-jail-scraper:latest