about | installation | usage | testing |

Ymir can be used to generate project boilerplate which combines fabric, boto, and puppet to create, setup, provision and control EC2 instances and, to a lesser extent, Elasticbeanstalk instances. Think of it as "glue code" that can help lay out project skeletons and give you a concise and relevant vocabulary for service description. Is it like ansible? Or maybe packer? How does it compare with vagrant or cloud formation or terraform? Why doesn't it use salts or chefs? OMG, I don't know! It seems useful though, and you might think so too. It's designed to (at least theoretically) help with creation, provisioning, etc in general regardless of the instance OS, but realistically some parts of it are specific to Ubuntu running on EC2.

• The Boto library is used for AWS automation (ex: creating EC2 instance from AMI)
• The Puppet DSL describes how the instances will be setup (ex: installing system packages)
• The Fabric library / framework is used for command execution on local and remote hosts

Ymir is "opinionated" in the sense that it mainly supports puppet for structured provisioning (or perhaps fabric for more adhoc provisioning) and it encourages daemonizing your services with supervisord. I'm not religious about any of these points though, and for instance the day when I find a big pile of prior art in the form of a chef script with no puppet equivalent then ymir will start to support chef.

And yes, ymir is obviously Yet Another Devops Tool in a rapidly expanding and sometimes confusing toolbox which may or may not make you sigh in frustration. Probably ymir configuration will eventually supprt "compiling" to terraform or cloudformation.

Prerequisites: A simple python setup for your platform, basically python, python-dev, & python-virtualenv. For debian you can use a one-liner like sudo apt-get install python python-dev python-virtualenv.

Installation:

  $ git clone https://github.com/mattvonrocketstein/ymir.git
  $ cd ymir
  $ virtualenv venv
  $ source venv/bin/activate
  $ pip install -r requirements.txt
  $ python setup.py develop

To test if it worked, try running ymir version.

Suggested software: * Puppet metadata validator

1 Before ymir can work with boto, you'll need to have boto setup (unless you use multiple amazon accounts, this only happens once). Follow the instructions here to setup your aws_access_key_id and aws_secret_access_key

2 Think a bit about how you want to setup your AWS keypairs for accessing the service. Some people may want one keypair per app, others may want one keypair per service and you may or may not want to change your keypairs per environment. You can create a keypair by issuing the following command:

    $ ymir keypair <app_name>

3 Initialize a new service with ymir. This will create a new folder with your project skeleton:

    $ ymir init demo_service

4 Customize your service description by making changes to the boilerplate demo_service/service.json (this file was created in step 3). The service.json file controls important service metadata like the service name, the base AMI, the EC2 instance size, and how health checks should be performed. Most of the options are illustrated in the default service.json. In particular you will want to change the name, key_name, and pem fields at minimum. For more detailed information about the fields and what they do, see the service description section of this documentation.

5 Customize your service implementation by making changes to the puppet provisioning code in the demo_service/puppet directory (this directory was created in step 3). Puppet is the best place to install system packages for your service, add daemons, configure users, etc etc. Eventually ymir may support chef or shell, but at the moment if you want to invoke that kind of provisioning you'll have to invoke it with puppet (or fork ymir and write some python code.. pull requests are welcome!). For more detailed information about common provisioning tasks with puppet, see the service implementation section of this documentation.

6 At this point you have a pretty good starting place for infrastructure as code: everything is declarative, and the approach leaves you with a lot of descriptive power and flexibility. Now it's time to actually test out service operations. If you're willing to write code you can define new operations of any kind, but you get lots of basic operations out of the box. To show a list of default operations, make sure you are in the service directory and run this command: fab -l. For instance, to build out your service based on the service.json file, run $ fab create.

Service operations are "command and control" helpers for your service. Operations are always fabric commands which are invoked from your local environment (or from a buildbot like jenkins) but are executed on your AWS-based service. Authentication with the remote side happens transparently and automatically, provided your keypair is setup correctly in your service description. To execute operations, cd into the directory for your your service and type fab operation_name. If you're willing to write code and especially if you're familiar with python and fabric, you can define new operations of any kind at any time. By default though, several operations are already defined for you automatically and they are described below.

Service descriptions are structured service metadata which you can find stored in service.json files. Particularly important fields which you will definitely want to override are described in the paragraphs below.

The security_groups field defines which AWS security groups this service will belong to. Ymir may eventually contain helpers for building security groups, but at the moment these should probably be constructed by hand in advance.

Both the setup_list and provision_list fields both describe a list puppet files which will be invoked in standalone-mode on the remote host, in the order they are listed. For more information on the difference between setup and provisioning, please see this section of the service operations documentation.

The key_name and pem fields are critical for authenticating with your service to complete updates, etc. The key_name field refers to a named AWS key, and the pem field should be a path that points to your corresponding AWS private key file. If you don't already have a key and pem file, you can create this data using ymir: see this section of the usage-overview documentation.

Service implementation is currently achieved with puppet code, but eventually ymir might support chef or shell. Pull requests welcome! Until this feature is added, invoking shell or chef from puppet may be the best option.