This is a demo created for a lightning talk designed to show off a number of things for a talk I gave at kiwipycon 2013. The talk was "The goodies that Zope begat" and highlighted a number of tools created or used in the Zope/Plone communities that might be useful Python developers. The lightning talk is the 1st in this youtube video.

I often start with a story of what I want to create which I write down in scenario form similar to this

User talks to a pirate parrot
   user opens piratechatapp
   user enters "pyramid_sockjs is cool"
   user hits "send"
   user sees "pyramid_sockjs is cool"

It's often not all the details of the interactions and not in any particular syntax. Next I start building the app or give this spec to someone else to build. The idea is for them to turn this into a "real" test as they build the app.

This is a simple chameleon template that mainly does its work in realtime by using sockjs.

./piratechat/app.pt

SockJS is a nice library that wraps both AJAX and websockets to create a bidirectional socket compatible with older browsers and networks. The template shows a simple form and some JS that sets up a connection back to the server.

./piratechat/app.py

Pyramid is combined with pyramid_sockjs to create a very simple-to-write realtime web server. pyramid_sockjs has advantages over Tornado in that it can run on the same port as normal http connections, and Pyramid is a great micro framework that won't hamper you if you decide to also create bigger projects with it.

class MapSession(Session):

     def on_message(self, line):
         print line
         response = urllib.urlopen("http://isithackday.com/arrpi.php?"+urllib.urlencode({'text':line})).read()
         print response
         self.send(response)

Since we expect chatting to a pirate parrot to be widely popular, we want an asynchronous IO-based server to solve the 10k problem. The 10k problem is how to handle 10k connections on a single machine without paying the overhead of 10k threads. This style of network library has been made popular by libraries such as NodeJS, Tornado and Twisted. Gevent solves this problem elegantly by not requiring you to change your code much, or work in awkward callbacks. Gunicorn allows us to replace threaded workers with gevent workers transparently so our code doesn't change at all.

This can be seen in ./paster.ini

[app:main]
use = egg:piratechat#app

[server:main]
use = egg:gunicorn#main
host = 0.0.0.0
port = 8080
workers = 1
worker_class = gevent

You'll notice we have a single worker, since asyncronous IO uses a single thread and takes control of which code gets control itself.

Since gevent requires libev which isn't normally installed on most platforms and we want a simple install experience for everyone, this demo uses buildout. Buildout installs, compiles and downloads everything you need to run complex apps and allows you to glue them together so you don't have to give complex install instructions.

The install instructions for a buildout-based application are always (note this doesn't change for any buildout-based application):

virtualenv .
bin/easy_install -U setuptools
bin/python bootstrap.py
bin/buildout

which will create your scripts in ./bin and things built in ./parts. In this case we also created a virtualenv to ensure our buildout was completely isolated from the system Python in case of package conflicts.

To run

bin/pserve paster.ini

or to run the gunicorn/gevent version

bin/gunicorn_paster paster.ini

A buildout is made up of parts, and what each part does is determined by a recipe and its part definition. A recipe is a package that is downloaded from pypi. For example the following gunicorn part

[gunicorn]
recipe = zc.recipe.egg
eggs =
  ${gevent:egg}
  gunicorn
  pyramid_sockjs
  piratechat
interpreter=mypy

is the buildout equivalent of

virtualenv .
bin/pip install gunicorn pyramid_sockjs piratechat
mv bin/python bin/mypy

Except that in addition it will also include a custom compiled version of gevent which is built in another buildout part.

Acceptance test driven development means that you take your original scenarios and turn them into acceptance tests. As you work on the product you turn the text into working test actions and assertions. In this case we will also need a tool that executes our tests in a real browser capable of javascript and websockets. RobotFramework is the perfect fit. Written in Python, with a modular library system, it can used Selenium to drive real browsers such as Firefox. Its simple test language allows us to turn very readable statements into actual tests.

In this case, your test_app.robot file includes

*** Test Cases ***

User talks to a pirate parrot
   user opens piratechatapp
   user enters "pyramid_sockjs is cool"
   user hits "send"
   user sees "pyramid_sockjs be cool"

*** Keywords ***

user opens piratechatapp
  go to  ${APP_URL}

user enters "${line}"
   Input Text  line  ${line}

user hits "${button}"
   Click Button  ${button}

user sees "${text}"
   Page should Contain  ${text}

Here we are using Selenium2 keywords but robot comes with many other libraries to test almost anything.

RobotFramework comes with its own test runner which we can use to run against a live server

bin/pybot piratechat/tests/test_app.robot

This will create logs of the run including screenshots and html of the pages where failures occur.

./robot_report.html

To run fully automated repeatable tests we need to setup and teardown our environment between tests. This is best done by integrating with Python's UnitTest framework via the RobotSuite tool.

We've included running the tests with two tools: zope.testrunner

bin/test
#TODO get this working

or pytest

bin/py.test .
#TODO get this working

Now that we have a working and tested app it's time to make it pretty. A frontend designer delivers some wonderful HTML and CSS; however it's still a work in progress. Rather than wait until they are finished, we decide to use WSGI middleware Diazo to theme our application rather than pull apart the HTML and templatise it.

#TODO

A community has grown around the pirate parrot and we want to have a blog and community forums with our app. We need a CMS. Plone is a great easy to use CMS written in python. We've decided to use WSGI middleware to combine our app into a part of our Plone site.

#TODO

We've decided to add a new feature that tracks what everyone enters. We just want something simple but still transactional and we don't want to both installing a whole relational database. We decide to use the ZODB

#TODO

We decide we want our application to be pluggable. We might like to have other services other than the speak-like-a-pirate translator. So we define a ITranslator service interface.

#TODO

We will adjust our URLs so you can share them with others. After you type in your name, you will get your own pirate chat room, and see all the recent chat in the room.

#TODO