GitHub

Dianping_Crawler

Tested on Ubuntu 14.04 with python2.7 and firefox46

Usage

Install environmental requirements by sudo bash install.sh
Set IDs in inputfile.txt, one dianping ID each line.
If it's the first time you run the crawler, set an ID to status.txt to start with. This ID should be in inputfile.txt. Afterwards the crawler will automatically record its status.
To start crawling use python2.7 dianping_u_script.py

Name		Name	Last commit message	Last commit date
Latest commit History 8 Commits
dianping_u_utils		dianping_u_utils
documents		documents
README.md		README.md
dianping_u_checkins_crawler.py		dianping_u_checkins_crawler.py
dianping_u_checkins_crawler.pyc		dianping_u_checkins_crawler.pyc
dianping_u_crawler_main.py		dianping_u_crawler_main.py
dianping_u_crawler_main.pyc		dianping_u_crawler_main.pyc
dianping_u_fans_crawler.py		dianping_u_fans_crawler.py
dianping_u_fans_crawler.pyc		dianping_u_fans_crawler.pyc
dianping_u_follows_crawler.py		dianping_u_follows_crawler.py
dianping_u_follows_crawler.pyc		dianping_u_follows_crawler.pyc
dianping_u_profile_crawler.py		dianping_u_profile_crawler.py
dianping_u_profile_crawler.pyc		dianping_u_profile_crawler.pyc
dianping_u_reviews_crawler.py		dianping_u_reviews_crawler.py
dianping_u_reviews_crawler.pyc		dianping_u_reviews_crawler.pyc
dianping_u_script.py		dianping_u_script.py
error_log.txt		error_log.txt
geckodriver.log		geckodriver.log
inputfile.txt		inputfile.txt
install.sh		install.sh
starting_time.txt		starting_time.txt
status.txt		status.txt

farrellsc/DianpingCrawler

Folders and files

Latest commit

History

Repository files navigation

Dianping_Crawler

Usage

About

Resources

Stars

Watchers

Forks

Languages