Skip to content
forked from icaicai/xspider

根据配置进行页面的抓取和分析的网络爬虫

Notifications You must be signed in to change notification settings

sumsung007/xspider

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

xspider

xSpider 是基于gevent的一个网络抓取程序/库。

它是可配置的,可根据配置规则来抓取网页并抽取其中的内容。

还可通过插件来对抽取后的内容进行自定义的处理。

Example

from xspider.console import Console

if __name__ == '__main__':
    args = sys.argv
    if len(args) > 1:
        path = args[1]
    else:
        path = './projs/'
    c = Console()
    c.init(path)
    c.run()

About

根据配置进行页面的抓取和分析的网络爬虫

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 99.6%
  • HTML 0.4%