爬虫

一个貌似很健壮的爬虫

TODO

Name		Name	Last commit message	Last commit date
Latest commit History 10 Commits
scheduler		scheduler
spiders		spiders
test		test
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
__init__.py		__init__.py
application.py		application.py
config.py		config.py
crawler.py		crawler.py
database.py		database.py
downloader.py		downloader.py
exception.py		exception.py
logmanager.py		logmanager.py
monitor.py		monitor.py
options.py		options.py
proxy.py		proxy.py
requirements.txt		requirements.txt
threadPool.py		threadPool.py
util.py		util.py
webPage.py		webPage.py