Skip to content

hoelzro/giraffe

 
 

Repository files navigation

Giraffe

Image pipelines are hard; let's go ride bikes!

This is partly a reaction to looking at the imgix service, which is "not just imagemagick on ec2", and saying that "imagemagick on ec2" sounds an awful lot like something I actually want.

travis ci build status

Why call it Giraffe?

I wanted a name with a soft G sound and I didn't have any better ideas. I'm actively soliciting suggestions for a better name.

About

giraffe is an image processing proxy with the immediate goal of being placed between an S3 bucket and a cloudfront distribution.

giraffe allows you to store your high-quality original images on S3 and do any resizing, cropping, filters, etc on demand by simply adding query parameters to the URL for the original image.

Overview of architecture:

You store original images in an s3 bucket. Something like:

s3://media.example.com/profile_pictures/1.jpg

You set up giraffe in ec2 behind an elb, point dns (e.g.: images.example.com) at it, and configure it to point to your bucket.

Finally set up a new cloudfront distribution (e.g.: hash.cloudfront.net) to sit in front of images.example.com and be sure to set it to forward query strings to origin.

Example

Assuming your image 1.jpg above is a 1080p image and you'd really like to just display a 100x100 thumbnail next to user_1's comments you can do so easily by simply including the following in your markup:

<img src="http://hash.cloudfront.net/media.example.com/profile_pictures/1.jpg?w=100&h=100"
     alt="profile pic" title="user_1" />

The request will hit cloudfront (at hash.cloudfront.com), which will turn around and hit your origin at images.example.com. giraffe will check for an image /profile_pictures/1.jpg, if that exists it'll then check for the requested size image at /cache/profile_pictures/1_100_100.jpg in s3://media.example.com. If it finds it it will simply return it, otherwise it'll use the original to generate the cache prefixed resized version.

Supported URL Parameters

Resizing

  • w, h
  • fit: controls how the output image is fitted to its target dimensions. Valid values for fit include:
  • crop (resize to fill width and height and crop any excess)
  • liquid (resize with liquid rescaling / content-aware resizing / seam carving)
  • crop: controls how the input is aligned when fit=crop. Valid values for crop include:
  • : crop to the center
  • top (TBD)
  • bottom (TBD)
  • left (TBD)
  • right (TBD)
  • flip (flip horizontally flip=h, vertically flip=v or both flip=hv)
  • rot (rotate, 1-359 degrees)

Setup

Dependencies

At a system level you'll need:

  • Python (2.7, 3.3+, or pypy)
  • ImageMagick (remember to use --with-liblqr if you want to be able to use content-aware resizing)

For deployment with Gunicorn you may also want libev.

Configuration

You need to set the following environment variables for giraffe to work properly:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY

Development

mkvirtualenv giraffe
pip install -r requirements.txt

Testing

nosetests --logging-clear-handlers

Or to test on multiple versions of Python:

tox

Deployment

Check out install.sh

TODO

  • documentation of camo functionality (proxying insecure image URLs like atmos/camo
  • documentation of placeholder functionality
  • support more imgix functionality
  • better caching: memcache caching of images is problematic; memcache complains about caching values over 1MB and it is sadly quite easy to do.
  • Use ec2-metadata and AWS roles to distribute S3 access tokens

Sites

  • Threadless.com
  • Typetees.com

About

NO TIME TO NAME THINGS

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 97.5%
  • Shell 2.5%