Skip to content

RyanZhu1024/lagou_crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

lagou_crawler

a python web crawler based on scrapy to grab data from lagou.com

Usage

  1. install mongodb, run it with wiretiger engine
  2. install python2.7, scrapy
  3. run scrapy crawl main_menu first
  4. run scrapy crawl jobs second
  5. run scrapy crawl details third
  6. fork it if you want to extend the project, like get all job details, which is I'm going to do next.

About

a python web crawler based on scrapy to grab data from lagou.com

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages