a python web crawler based on scrapy to grab data from lagou.com
- install mongodb, run it with wiretiger engine
- install python2.7, scrapy
- run
scrapy crawl main_menu
first - run
scrapy crawl jobs
second - run
scrapy crawl details
third - fork it if you want to extend the project, like get all job details, which is I'm going to do next.