We were interested in comparing Gevent with Tornado to see which would better suit our needs. The starting point was a benchmark done by Nicholas Piël in 2010 showing that Tornado had excellent performance followed closely by Gevent.

We did a very quick and non-rigorous benchmark in 2013-01-05. Nevertheless we are making the code and results available in case this can be useful elsewhere.

First we tested a linux client generating HTTP traffic towards a MacOSX running the server backend. This test was run over the corporate wi-fi network. Then, we recreated the same test scenario in two different sets of Amazon instances to cross-validate the performance measured.

In the client side, we used wrk, weighttp and locust to generate traffic.

The test cases where basically:

• the HTTP server just responds a "hello world" to a GET request.
• the HTTP server queries a backend Virtuoso triple-store database
• the HTTP server queries a backend REDIS database

In the tests we used the following hardware configuration:

• (corporate) Linux running Fedora 16, Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz 8Gb RAM, cache size 4096 KB
• (corporate) MacOSX lion 10.7.5 - 2Ghz Intel Vore i5 8Gb RAM, L2 Cache (per Core) 256 KB
• (Amazon) Linux Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-31-virtual x86_64), Intel(R) Xeon(R) CPU E5645 @ 2.40GHz, cache size 12288 KB

The test cases were:

• tornado_lan_flask_hw - Tornado returning a simple "Hello World" for GET / with Flask in corporate network

• tornado_lan_sock_hw - Tornado returning a simple "Hello World" for GET / with pure sockets in corporate network

• tornado_virt_lan - Tornado doing a query in Virtuoso (all servers in the same machine) in corporate network

• tornado_virt_s3 - Tornado doing a query in Virtuoso (all servers in the same machine) in Amazon Srv3

• tornado_virt_s1s2_pycurl - Tornado doing a query in Virtuoso (servers in different machines) in Amazon Srv1-Srv2

• tornado_virt_s1s2_native - Tornado doing a query in Virtuoso (servers in different machines) in Amazon Srv1-Srv2 (without PyCurl)

• tornado_virt_pypy - Tornado doing a query in Virtuoso (servers in different machines) in Amazon Srv1-Srv2 (over Pypy)

• tornado_inc_load - Tornado incremental load

• gevent_lan_flask_hw - Gevent returning a simple "Hello World" for GET / with Flask in corporate network

• gevent_lan_sock_hw - Gevent returning a simple "Hello World" for GET / with pure sockets in corporate network

• gevent_pure_wsgi - Gevent returning a simple "Hello World" for GET / with pure WSGI in corporate network

• gevent_virt_lan - Gevent doing a query in Virtuoso (all servers in the same machine) in corporate network

• gevent_virt_s3 - Gevent doing a query in Virtuoso (all servers in the same machine) in Amazon Srv3

• gevent_virt_s3_alt - Gevent (alternate implementaion) doing a query in Virtuoso (all servers in the same machine) in Amazon Srv3

• gevent_virt_s1s2 - Gevent doing a query in Virtuoso (servers in different machines) in Amazon Srv1-Srv2

• gevent_virt_s1s2_alt - Gevent doing a query in Virtuoso (servers in different machines) in Amazon Srv1-Srv2

• gevent_inc_load - Gevent incremental load

• gevent_inc_load_alt - Gevent incremental load (alternate implementaion)

The table below depicts the average results for comparison:

The following table presents how "request/s" evolves as the client load increases. The columns represent the number of parallel connections in the client that generates the load.

The attached Benchmarks.md file contains dumps of the raw tests executed.

Tornado shows consistently a better performance than Gevent considering just "requests/s", and responds better when we increase the number of requests, thus, it is a good hint that will scale better. However, the code produced with Gevent was cleaner and more readable than the code produced with Tornado.

When using Tornado, the adoption of the pycurl client (instead of the native client) provides an enhancement in the performance.

The use of Flask significantly reduces the performance of both Tornado or Gevent.

Comparing tornado_virt_s3 with tornado_virt_pypy showed that PyPy adoption did not contributed to any performance enhancement. We were unable to run PyPy with Gevent.

We did not have consistent results of which is the best http client for Gevent: either the patched "requests" lib or "geventhttpclient". When the servers are separate (more realistic use case), patched "requests" performed better than "geventhttpclient".

We did some other tests accessing a backend Redis server, instead of Virtuoso. The source code is also published in this project, but we did not reported here the detailed results of those tests. When querying the Redis server using a synchronous driver the results of Tornado and Gevent had equivalent performances, however neither was stable.

DISCLAIMER: This is a quick-and-dirty benchmark for us to have a glimpse of how to code in Tornado in comparison with how to code in Gevent, and to have a coarse grain idea of performance. A serious benchmark would need much more repetitions under more controlled conditions.

The team members that took part in the testing efforts were:

• Guilherme Machado Cirne
• Ícaro Medeiros
• Rodrigo Dias Arruda Senra
• Tatiana Al-Chuery Pereira Martins

• [1] http://nichol.as/benchmark-of-python-web-servers
• [2] https://github.com/wg/wrk
• [3] http://redmine.lighttpd.net/projects/weighttp/wiki
• [4] http://locust.io/
• [5] https://github.com/bobhancock/Pycon-2012-Parallelism-and-Concurrency