yanniey / Scrapy_livingsocial_chicago Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

Use Scrapy (Python) to scrape deals info from LivingSocial's Chicago page and store the crawled data into a Postgre database

0 stars 0 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
living_social		living_social
PSQL screenshot.png		PSQL screenshot.png
README.md		README.md
items.json		items.json
output.png		output.png
requirements.txt		requirements.txt

Repository files navigation

Data scraping with Scrapy (Python)

5/14/15

Use Scrapy to scrape deals info from LivingSocial's Chicago page

Steps:

Config Scrappy spider to crawl and parse HTML
Set up Postgre database to store the parsed data
Create a pipeline to connect parsed data and dtabase with SQLAlchemy ORM

Result:

PSQL db:

Json output:

About

Use Scrapy (Python) to scrape deals info from LivingSocial's Chicago page and store the crawled data into a Postgre database

Report repository

Releases

No releases published

Packages

No packages published

Languages