2023/09 updated package versions and made it run with Python 3.11 and Flask 2.0. Email functionality not tested.

Application prototype helps tracking dairy farm’s biological/chemical waste water cleaning system’s health and provides email notifications for doing repeated manual measurements. Cleaning system used is Atomar Oy “Milkilo” http://web.archive.org/web/20180828210118/http://atomar.fi/panospuhdistamo.html.

• Password protected Web UI for logging/viewing data
• Email notifications with configurable alert intervals for different measurement types

Built with

• Python Flask web framework
  • Flask-Login extension for simple single user authentication
  • Jinja2 templating engine
  • Bootstrap HTML UI toolkit with Flask-Bootstrap extension
  • WTForms HTML form library for input validation
• Python Dataset SQL Database wrapper (tested on SQLite and PostgreSQL)
• Mailgun for sending emails

Different measurement types are defined in `model.py` with ranges that are validated by UI.

entry_model = \
            {"silt_active_ml_per_l":
             {"finnish": "Aktiivilietemittaus", "min": 0, "max": 1000,
              "description": "aktiivilietteen mittaus (ml per litra)",
              "default_interval": 14},
             "silt_surplus_removal_l":
             {"finnish": "Poistopumppaus", "min": 1, "max": 1000,
              "description": "ylijäämälietteen poistomäärä litroina"},
             "pump_usage_hours":
             {"finnish": "Pumpun käyttötunnit", "min": 1, "max": 9999999,
              "description": "pumpun käyttötuntilaskurin lukema"},
             "water_quality":
             {"finnish": "Kirkasvesinäyte", "min": 1, "max": 5,
              "description":
              "kirkasveden laatu asteikolla 1-5 (1 on parhain)"},
             "ferrosulphate_level_percent":
             {"finnish": "Ferrosulfaatin määrä", "min": 0, "max": 100,
              "description": "ferrosulfaatin määrä sentteinä",
              "default_interval": 14},
             "ferrosulphate_addition_kg":
             {"finnish": "Ferrosulfaatin lisäys", "min": 1, "max": 1000,
              "description":
              "Ferrosulfaatin määrän lisäys kiloina."}}

• [X] +User auth by env token, flask-login
• [X] Dataset and sqlite
• [X] Views
• [X] WTForms classes & validation for entries
• [X] Schedule flask-mail notifications
• [X] Serving on heroku
• [X] Backups
• [ ] SSL
• [ ] Secret token

Code was written in quickly in few days and was not tested by design. It wouldn’t be convenient to build much bigger programs than this without automated testing facilities in place.

Originally developed on Ubuntu 14.04 and OSX 10.10, Python 3.4. Following notes were written back then..

Distribution provided pre-requirements:

• Python with development package
• PostgreSQL development package libpq-dev

sudo apt-get install python3-pip python3-dev libpq-dev

Install project dependencies pip install -r requirements.txt.

Initialize tables python server.py init_db. Database can be cleared with command python server.py drop_db when necessary.

Application defaults to password ‘topsecret’ and throws errors when running email notification checker if following variables are not set:

FLASK_APP_PASSWORD="somethingHard_to-guess"
MAILGUN_KEY='key-xxxblablaetcxxx'
ADMIN_EMAIL="foo@example.com"
FLASK_APP_EMAIL="bar@example.com"
SERVER_ADDRESS='http://your-awesome-domain.net'

Command python server.py dev defaults to localhost:5000. Changing host and port for exposing server for external requests (might be dangerous if debugger is enabled) can be accomplished by running python server.py dev -h 0.0.0.0 -p 80.

Heroku free plan was used for hosting but free plan got removed after 2022, should use Fly.io instead.

With Heroku Toolbelt installed and account configured:

1. Clone repository git clone <repository url>
2. Create Heroku app heroku create <application_name>
3. Add free PostgeSQL addon heroku addons:create heroku-postgresql:hobby-dev
4. Push local repository to Heroku git push heroku master
5. Initialize database heroku run python app/server.py init_db
6. Setup environment variables from previous step
   • For Heroku, config variables are set with command heroku config:set <VARIABLE_SETTING>.
7. Access application at http://<application_name>.herokuapp.com

When continuing work on deployed Heroku app on other machine etc, skip steps 2-3 and instead set remote to existing Heroku git repository heroku git:remote -a <application_name>.

Optionally setup cronjob somewhere to keep node running during daytime by adding following line to cron:

*/8 06-21 * * * curl http://<app_name>.herokuapp.com

See Procfile (Heroku configuration file) for hints, at least logging won’t work by default via run.py so it needs work.

Notifications for doing scheduled measurements are sent to addresses declared by environment variables ADMIN_EMAIL and FLASK_APP_EMAIL. Database stores only date, so hourly running check_notifications script may send notifications during night, which might cause undesired side effects.

Heroku Scheduler can be used to send notifications on daily basis, but it requires linking a credit card to Heroku account as scheduled tasks going over the given 750 monthly free dyno hours will require payment. To enable notifications via Heroku Scheduler addon:

1. heroku addons:create scheduler
2. Open scheduler web UI heroku addons:open scheduler
3. Setup new scheduler command python app/server.py check_notifications

1. Run crontab -e or edit /etc/anacrontab
2. Add rule * 7 * * * heroku run --app <app_name> python app/server.py check_notifications

(didn’t test this yet, but should work, maybe with minor fixing)

Notification script python app/server.py check_notifications can be scheduled simply in cron or anacron. Machine should have environment variable DATABASE_URL set to application database. Heroku database url can be queried with heroku config.

(didn’t test this yet, but should work, maybe with minor fixing)

App serves CSV file of measurements via the public /backup endpoint. This can easily be called by cron (or anacron on desktop). Example for backing up twice a month:

1. Create directory for backups eg. mkdir -p /home/user/backups/measurements
2. Modify crontab by crontab -e and add rule

0 0 1/16 * * cd /home/user/backups/measurements && wget --trust-server-names http://<app_name>.herokuapp.com/backup

Wget --trust-server-names option is required for keeping the timestamped filename. Resulting backup files are in form measurement_backup_2015-09-20.csv.