An attempt to centralize the configuration and building of application environments at eLife, locally (Vagrant) and remotely (AWS).

Download:

git clone ssh://git@github.com/elifesciences/builder

Install:

./update.sh

Fix any missing pre-requisites and call ./update.sh again until you see the happy

all done

message. Updating in the future is as simple as:

git pull
./update.sh

Your project file is located at ./projects/elife.yaml. This file describes all eLife projects that can be built and their environments. See here for more project file documentation.

'configuration' also exists in ./src/buildercore/config.py if you're a builder dev.

After successfully installing and configuring builder, launching a Vagrant instance is a good test that all is working correctly.

The Vagrantfile can build any project, you just need to tell it which one:

$ vagrant up
You must select a project:

1 - journal--vagrant
2 - api-gateway--vagrant
3 - ...
>

... or it can be done with environment variables:

$ PROJECT=journal vagrant up

To execute the Python part of the Vagrantfile with Docker, create this flag:

touch .use-docker.flag

Note: if you with to use a private key not in ~/.ssh/id_rsa, you can customize the SSH key path.

Note: if you wish to use a hypervisor other than virtualbox, you can use the vagrant-mutate plugin to rebuild the ubuntu/trusty64 box for your own hypervisor. See the vagrant and virtualbox documentation.

Formula projects are cloned to the local cloned-projects directory. Changes will be picked up from there and branches can be used as well.

After starting (vagrant up) and login in to (vagrant ssh) the VM, run the following command to change the remote commit or branch:

$ set_local_revision $commitOrBranch

Provisioning (vagrant provision or sudo salt-call state.highstate) will then checkout the particular commit or branch.

The other half of the builder project is the ability to create and manage AWS resources. This is controlled with the "bldr" script:

$ ./bldr -l

Will list all builder tasks found in src/. These tasks are just Python functions.

builder relies on a ~/.aws/credentials file containing configuration for accessing your AWS account.

A master-server instance must exist before project instances can be brought up. See here for a walkthrough.

To launch a project backed by a code repository to AWS:

$ ./bldr deploy
// or specify project and environment
$ ./bldr deploy:journal,prod

To launch a instance of any project to AWS, use:

// or specify project and environment
$ ./bldr launch:api-gateway,prod

To ssh into one of these machines:

$ ./bldr ssh:journal--prod

General:

• project files

AWS:

• feature: write keypairs to S3
• creating a master server
• caveats
• EKS

Troubleshooting:

• ssh-agent
• vault

Development

• adding projects
• technology
• salt
• testing

The builder project is MIT licenced.

The builder project was GPL3 licensed until this commit on 2016-07-13.