Cloudwatch-to-Graphite (leadbutt) is a small utility to take metrics from CloudWatch to Graphite.
Install using pip:
pip install cloudwatch-to-graphite
Cloudwatch-to-Graphite uses boto, so make sure to follow its configuration instructions. The easiest way to do this is to set up the AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
environment variables.
If you have a simple setup, the easiest way to get started is to set up a config.yaml. You can copy the included config.yaml.example. Then just run:
leadbutt
If you have several configs you want to switch between, you can specify a custom configuration file:
leadbutt --config-file=production.yaml -n 20
You can even generate configs on the fly and send them in via stdin by setting the config file to '-':
generate_config_from_inventory | leadbutt --config-file=-
There's a helper to generate configuration files called plumbum
. Use it like:
plumbum [-r REGION] [-f FILTER] [--token TOKEN] template namespace
Namespace is the CloudWatch namespace for the resources of interest; for example AWS/RDS
. The template is a Jinja2 template. You can add arbitrary replacement tokens, eg {{ replace_me }}
, and then pass in values on the CLI via --token
. For example, if you called:
plumbum --token replace_me='hello, world' sample_templates/rds.yml.j2 AWS/RDS
You would get all instances of {{ replace_me }}
in the templace replaced with hello, world
.
You can pass simple key=value
filters in to plumbum
; be aware of the limitations:
- the filters run against whatever the AWS API has returned; if you have a lot of objects of whatever type, expect the API request to take a while.
- they work only against object attributes and tags returned by the API. For example, RDS and ELB objects can be tagged, but as getting the tags is a per-object subrequest;
plumbum
does not do those, so you can only filter on the object attributes.
Example: plumbum -f Name=my-dev-instance sample_templates/ec2.yml.j2 ec2
If your graphite server is at graphite.local, you can send metrics by chaining with netcat:
leadbutt | nc -q0 graphite.local 2003
Or if you want to use UDP:
leadbutt | nc -uw0 graphite.local 2003
If you need to namespace your metrics for a hosted Graphite provider, you could provide a custom formatter, but the easiest way is to just run the output through awk:
leadbutt | \
awk -v namespace="$HOSTEDGRAPHITE_APIKEY" '{print namespace"."$0}' | \
nc -uw0 my-graphite-provider.xxx 2003
Set the Formatter
option to set the template used to generate Graphite metric names. I wasn't sure what should be default, so I copied cloudwatch2graphite's. Here's what it looks like:
cloudwatch.%(Namespace)s.%(dimension)s.%(MetricName)s.%(statistic)s.%(Unit)s
Enhanced Monitoring has two options for formatting, because some of the metrics contain a list of dictionaries. This is not required, but if your list contains more than one dictionary it will overwrite. In the example below eth0
would be used as ListCategory
in formatter.
`json "network":[ { "interface":"eth0", "rx":92.33, "tx":1561.72 } ]
`cloudwatch.%(Namespace)s.%(Dimension)s.EnhancedMonitoring.%(MetricName)s.%(Statistic)s.%(Unit)s cloudwatch.%(Namespace)s.%(Dimension)s.EnhancedMonitoring.%(MetricName)s.%(ListCategory)s.%(Statistic)s.%(Unit)s
TitleCased variables come directly from the YAML configuration, while lowercase variables are derived:
- statistic -- the current statistic since
Statistics
can be a list - dimension -- the dimension value, e.g. "i-r0b0t" or "my-load-balancer"
The format string is Python's %-style.
What metrics are pulled is in a YAML configuration file. See the example config.yaml.example for an idea of what you can do.
Install requirements:
pip install -r requirements.txt
Running test suite:
make test
Verifying tests run over all supported Python versions:
tox
Cloudwatch-to-Graphite was inspired by edasque's cloudwatch2graphite. I was looking to expand it, but I wanted to use boto.