Skip to content

ewheeler/vaxtrack

Repository files navigation

Online monitoring and alert system integrating
vaccine stock data (national coldstore data),
delivery forecasts, and cold chain capacity
tools to avoid stock outs and vaccine spoilage/wastage.

Summary of features:
- uploading of UNICEF Supply Division forecast and allocation xls files
- uploading of WHO country stock data xls files
- automatic post-upload analysis to generate statistics and flag potential stock issues
- automatic email alerts of potential issues that emerge during analysis
- dynamic charts displaying stock levels, forecasts, and purchases
- ability to view historical perspectives (to see data-awareness/situation as of a certain date)
- uploading and manual entry of stock information by Country Office, partners, and/or governments
- ability to export data used to generate charts
- web interface for entering and editing translations (English and French)

Technology stack overview:
    Vaxtrack is a django project that uses Amazon AWS products heavily, as stock
    datasets can quickly become large and unweildy. Vaxtrack was originally
    intended to target IE6 web browsers, so charts were generated as images
    using matplotlib and then stored on S3. Thankfully, the client organization
    upgraded to IE8 in late 2010, so Vaxtrack has been altered to produce
    javascript charts with the dygraph library. IE8 compatibility has been achieved
    using FlashCanvas, a drop-in product for emulating the canvas tag.
    FlashCanvas may rarely cause IE8 to crash, so using Google's Chrome Frame
    extension to IE8 is strongly recommended -- and is now available as a plug-in
    that does NOT require admin privileges to install (http://www.google.com/chromeframe)
    Uploading, analysis, and email alerting are queued and executed via celery,
    which uses RabbitMQ for messaging. RabbitMQ is certainly overkill for this
    use-case, but is perhaps more reliable than some of the 'ghettomq' alternatives.
    South is employed for MySQL database migrations.

Summary of data storage:
- vaccine stock levels, forecasts, and purchases are stored in Amazon's SimpleDB
- user, country, product group, product, and upload information are stored in MySQL
- uploaded documents are stored in Amazon's S3
- CSV files for export are currently stored in local filesystem TODO move to S3

Wayfinding in the codebase (and notes about important files):
- country and product group models (and methods) are defined in models.py
      - countries can be reconciled by ISO 2 letter code, ISO 3 letter code,
        ISO English name, ISO French name, UN 3 number code, UNICEF SD code,
        UNICEF SD English name, and UNICEF SD French name.
      - product groups are used internally instead of products, because WHO
        stock level granularity is only to product groups
      - country, product group, and products are provided in vaxapp/fixtures
        as database fixtures in json format
- vaccine information in SimpleDB is accessed by helper API methods in vsdb.py
      - boto's class-based managers were not working when development began,
        so instead there is a dodgy approximation that should be replaced by
        boto's offering once it is fully implemented
      - nearly all SimpleDB items have the same structure that includes:
            date - date amount was observed or delivery was made as str(date) AKA date.isoformat()
            year - year from above date, as string for easier sorting
            amount - stock level, forecast amount, or purchased amount in doses
            type - indicates type of data:
                'CF' - country forecast from UNICEF SD country forecasting files
                'CO' - original country forecast from UNICEF SD country forecasts files
                'FP' - future delivery on PO from UNICEF SD allocations table files
                'UN' - unicef delivery from UNICEF SD allocations table files
                'FF' - future delivery on forecast from UNICEF SD allocations table files
                'SL' - stock level from WHO stock files or other stock level uploads
                'CS' - no amount, just country + product + type to keep track of known inventories
            country - country as ISO 2 letter code
            group - product group as slug
            product - product as slug
            activity - 'Supplementary' or 'Routine' accoring to UNICEF SD files
            initial - initial stock level at beginning of year (only for CFs)
- importing scripts are located in import_xls.py
    - import methods for UNICEF and WHO data may be called from a python
    interpreter (or ./manage.py shell) for an interactive version that allows
    for reconciliation of unknown country names and/or product group names.
    Any choices made during interactive reconciliation are saved as country-aware
    alternates, so that these specific reconciliations will be made automatically
    in future interactive sessions and also during web-uploaded imports.
    The code for matching and suggestion of alternatives lives in models.py and
    employs a number of techniques for guessing, including many hand-entered
    edge cases based on a corpus of UNICEF datasets' creative spellings. These
    reconciliation methods are also available in country-aware versions which
    base suggestions on a country's past stock history (will suggest products
    that the country has stocked in the past when faced with a similar, ambiguous
    product).
- uploaded documents are queued and handled by celery in tasks.py
- business logic for post-upload analysis is in analysis.py
    - CountryStock object creation
    - CountryStockStat object creation
        - dict-like stats are stored in embarassing Dicty and KeyVal models,
            but should be accessed through helper methods in CountryStockStat
    - Alert object creation, trigger email tasks
    - creation of csv files used by dygraph and export
- front-end is largely controlled by vaxapp/static/javascripts/vax-dev.js
    - front-end also makes use of jquery.sparklines.js and a patched version
        of jquerytool's tooltip.js (included)
    - css is mostly found in vaxapp/static/stylesheets/style.js, but additional
        styles for modules and forms exist in separate files
- translations are implemented using django best practices, which require
    management commands to compile po files after editing. rosetta is employed
    to offer translation editing via a web interface. see INSTALL for more notes
    some translations, such as country names and product group names, are handled
    by property methods on respective model.
- currently, the web interface shows a single 'anonymized' country. to un-anonymize,
    visit the gargoyle section of the admin site (/nexus) and disable the anonymous switch.
    note there are two versions of vax-dev.js: vax-dev.js and vax-dev-anon.js
    this is unfortunately the simplest way to allow switching between real and anonymous
- run ./manage.py runserver in one screen and ./manage.py celeryd -v 2 -B -s celery -E -l INFO in another

For more information:
    please contact me! this guide is certainly not comprehensive

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published