一个微型爬虫框架
spider.py 使用 gevent ，没有更新。。
crawler.py 使用 threading。所有的改动都集中在这个文件了。
url 抓取列表在 redis DB 1 里面
倒排索引在 redis DB 2 内，unicode 编码
forked from jude90/zhihu
luyajie/zhihu
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
知乎采集者
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- Python 99.0%
- Other 1.0%