A free, open source development platform for data curation, exchange, and publication.

• Table of contents
• Setup
  • Dependencies
  • Installation
  • Common Module
  • Environment Variables
    • Aether Kernel
    • Aether ODK Module
    • Aether UI
    • Aether CouchDB Sync Module
    • File Storage System
• Usage
  • Users & Authentication
    • Basic Authentication
    • Token Authentication
• Development
• Deployment
• Containers and services
• Run commands in the containers
  • Run tests
  • Upgrade python dependencies
    • Check outdated dependencies
    • Update requirements file

• git
• docker-compose

Return to TOC

git clone git@github.com:eHealthAfrica/aether.git && cd aether

Note: Make sure you have openssl installed in your system.

./scripts/generate-docker-compose-credentials.sh > .env

./scripts/build_aether_containers.sh && ./scripts/docker_start.sh

or

./scripts/docker_start.sh --build

IMPORTANT NOTE: the docker-compose files are intended to be used exclusively for local development. Never deploy these to publicly accessible servers.

127.0.0.1    kernel.aether.local odk.aether.local ui.aether.local sync.aether.local

Return to TOC

This module contains the shared features among different containers.

To create a new version and distribute it:

./scripts/build_common_and_distribute.sh

See more in README.

Return to TOC

Most of the environment variables are set to default values. This is the short list of the most common ones with non default values. For more info take a look at the file docker-compose-base.yml.

• DB_NAME: aether Database name.
• WEB_SERVER_PORT: 8100 Web server port.
• ADMIN_TOKEN: kernel_admin_user_auth_token to connect to it from other modules. It's used within the start up scripts.
• CSV_SEPARATOR: , fields separator in the CSV export format.

• DB_NAME: odk Database name.
• WEB_SERVER_PORT: 8102 Web server port.
• ADMIN_TOKEN: odk_admin_user_auth_token to connect to it from other modules. It's used within the start up scripts.
• AETHER_KERNEL_TOKEN: kernel_any_user_auth_token Token to connect to kernel server.
• AETHER_KERNEL_URL: http://kernel:8100 Aether Kernel Server url.
• AETHER_KERNEL_URL_TEST: http://kernel-test:9100 Aether Kernel Testing Server url.

• DB_NAME: ui Database name.
• WEB_SERVER_PORT: 8104 Web server port.
• AETHER_KERNEL_TOKEN: kernel_any_user_auth_token Token to connect to kernel server.
• AETHER_KERNEL_URL: http://kernel:8100 Aether Kernel Server url.
• AETHER_KERNEL_URL_TEST: http://kernel-test:9100 Aether Kernel Testing Server url.

• DB_NAME: couchdb-sync Database name.
• WEB_SERVER_PORT: 8106 Web server port.
• ADMIN_TOKEN: sync_admin_user_auth_token to connect to it from other modules. It's used within the start up scripts.
• AETHER_KERNEL_TOKEN: kernel_any_user_auth_token Token to connect to kernel server.
• AETHER_KERNEL_URL: http://kernel:8100 Aether Kernel Server url.
• AETHER_KERNEL_URL_TEST: http://kernel-test:9100 Aether Kernel Testing Server url.
• GOOGLE_CLIENT_ID: generate_it_in_your_google_developer_console Token used to verify the device identity with Google. See more in https://developers.google.com/identity/protocols/OAuth2

(Used on Kernel and ODK Module)

• DJANGO_STORAGE_BACKEND: Used to specify a Default file storage system. Available options: minio, s3, gcs. More information here. Setting DJANGO_STORAGE_BACKEND is mandatory, even for local development (in which case "minio" would typically be used with the minio service).

• BUCKET_NAME: Name of the bucket that will act as MEDIA folder (mandatory).
• MINIO_STORAGE_ACCESS_KEY: Minio Access Key.
• MINIO_STORAGE_SECRET_KEY: Minio Secret Access Key.
• MINIO_STORAGE_ENDPOINT: Minio server url endpoint (without scheme).
• MINIO_STORAGE_USE_HTTPS: Whether to use TLS or not. Determines the scheme.
• MINIO_STORAGE_AUTO_CREATE_MEDIA_BUCKET: Whether to create the bucket if it does not already exist.
• MINIO_STORAGE_MEDIA_USE_PRESIGNED: Determines if the media file URLs should be pre-signed.

See more in https://django-minio-storage.readthedocs.io/en/latest/usage

• BUCKET_NAME: Name of the bucket to use on s3 (mandatory). Must be unique on s3.
• AWS_ACCESS_KEY_ID: AWS Access Key to your s3 account.
• AWS_SECRET_ACCESS_KEY: AWS Secret Access Key to your s3 account.

• BUCKET_NAME: Name of the bucket to use on gcs (mandatory). Create bucket using Google Cloud Console and set appropriate permissions.
• GS_ACCESS_KEY_ID: Google Cloud Access Key. How to create Access Keys on Google Cloud Storage
• GS_SECRET_ACCESS_KEY: Google Cloud Secret Access Key. How to create Access Keys on Google Cloud Storage

WARNING

Never run odk, ui or couchdb-sync tests against any PRODUCTION server. The tests clean up would DELETE ALL PROJECTS!!!

Return to TOC

Start the indicated app/module with the necessary dependencies:

./scripts/docker_start.sh [--kill | -k] [--build | -b] [--force | -f] name

Arguments:

• --kill | -k kill all running containers before start
• --build | -b kill and build all containers before start
• --force | -f ensure that the container will be restarted if it was already running
• name expected values: kernel, odk, ui, couchdb-sync or sync (alias of couchdb-sync). Any other value will start all containers.

This will start:

• Aether Kernel on http://kernel.aether.local and create a superuser $ADMIN_USERNAME with the given credentials.

• Aether ODK Module on http://odk.aether.local and create a superuser $ADMIN_USERNAME with the given credentials.

• Aether UI on http://ui.aether.local and create a superuser $ADMIN_USERNAME with the given credentials.

• Aether CouchDB Sync Module on http://sync.aether.local and create a superuser $ADMIN_USERNAME with the given credentials.

If you generated an .env file during installation, passwords for all superusers can be found there.

To start any app/module separately:

./scripts/docker_start.sh kernel          # starts Aether Kernel app and its dependencies

./scripts/docker_start.sh odk             # starts Aether ODK module and its dependencies

./scripts/docker_start.sh ui              # starts Aether UI and its dependencies

./scripts/docker_start.sh couchdb-sync    # starts Aether CouchDB Sync module and its dependencies

Return to TOC

Set the HOSTNAME and CAS_SERVER_URL environment variables if you want to activate the CAS integration in the app. See more in Django CAS client.

Other options are to log in via token authentication, via basic authentication or via the standard django authentication.

The available options depend on each container.

Return to TOC

The communication between Aether ODK Module and ODK Collect is done via basic authentication.

Return to TOC

The communication between the containers is done via token authentication.

In the case of aether-odk-module, aether-ui and aether-couchdb-sync-module there is a global token to connect to aether-kernel set in the required environment variable AETHER_KERNEL_TOKEN. Take in mind that this token belongs to an active aether-kernel user but not necessarily to an admin user.

Return to TOC

All development should be tested within the container, but developed in the host folder. Read the docker-compose-base.yml file to see how it's mounted.

To get started on building solutions on Aether, an aether-bootstrap repository has been created to serve as both an example and give you a head start. Visit the Aether Website for more information on Try it for yourself.

Return to TOC

Set the HOSTNAME and CAS_SERVER_URL environment variables if you want to activate the CAS integration in each container.

Set the AETHER_KERNEL_TOKEN and AETHER_KERNEL_URL environment variables when starting the aether-odk-module to have ODK Collect submissions posted to Aether Kernel.

If a valid AETHER_KERNEL_TOKEN and AETHER_KERNEL_URL combination is not set, the server will still start, but ODK Collect submissions will fail.

To check if it is possible to connect to Aether Kernel with those variables visit the entrypoint /check-kernel in the odk server (no credentials needed). If the response is Always Look on the Bright Side of Life!!! it's not possible to connect, on the other hand if the message is Brought to you by eHealth Africa - good tech for hard places everything goes fine.

This also applies for aether-ui and aether-couchdb-sync-module.

In the case of aether-couchdb-sync-module a valid GOOGLE_CLIENT_ID environment variable is necessary to verify the device credentials as well.

Return to TOC

The list of the main containers:

All of the containers definition for development can be found in the docker-compose-base.yml file.

Return to TOC

Each docker container uses the same script as entrypoint. The entrypoint.sh script offers a range of commands to start services or run commands. The full list of commands can be seen in the script.

The pattern to run a command is always docker-compose run <container-name> <entrypoint-command> <...args>

Return to TOC

This will stop ALL running containers and execute the containers tests.

./scripts/test_all.sh

To execute tests in just one container.

./scripts/test_container.sh <container-name>

or

docker-compose run <container-name> test

or

docker-compose run <container-name> test_lint
docker-compose run <container-name> test_coverage

The e2e tests are run against different containers, the config file used for them is docker-compose-test.yml.

Before running odk, ui or couchdb-sync you should start the needed test containers.

docker-compose -f docker-compose-test.yml up -d <container-name>-test

WARNING

Never run odk, ui or couchdb-sync tests against any PRODUCTION server. The tests clean up would DELETE ALL PROJECTS!!!

Look into docker-compose-base.yml, the variable AETHER_KERNEL_URL_TEST indicates the Aether Kernel Server used in tests.

Return to TOC

docker-compose run --no-deps <container-name> eval pip list --outdated

./scripts/upgrade_all.sh

This also rebuilds aether.common module and distributes it within the containers. Do not forget to include new containers in the file.

or

docker-compose run --no-deps <container-name> pip_freeze

In this case aether.common is not rebuilt.

Return to TOC