GitHub - yowenter/jd-scrapy

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
jd		jd
README		README
scrapy.cfg		scrapy.cfg

Repository files navigation

Author:wenter
Email:wenter.wu@gmail.com
Org:DaoCloud


This is a crawler for jd.com .

Requires:


Python2.7
Scrapy
BeautifulSoup


RUN in jd project dir:

-scrapy crawl jd-category


ps:
You can edit the settings.py .
  FILTER_CATEGORIES=['beauty']
  
That means the spider will crawl category 'beauty'. 
If you wanna crawl more, you can  add ,like this: FILTER_CATEGORIES=['beauty','car']