a light framework of crawler with gevent, requests, leveldb

1. multiple crawling task, use coroutine of gevent;
2. a url pool to manage urls (downloaded or to download), use leveldb;
3. a proxy pool to avoid blocked by the site while quering very frequently;

sudo python setup.py install

Sorry for no docs right now, just see the example or read the source code