Recipes Scraper

Using scraper

Install docker-compose
Clone this repo
Go to the directory with the repo and run docker-compose up -d from command line
Install scrapyd-client
Run scrapyd-deploy from command line
Schedule some spider to run with curl http://localhost:6800/schedule.json -d project=gustoqa -d spider=copykat executed from command line
You can view the result in RabbitMQ management panel at http://localhost:15672/
Images are stored in /stock directory inside the container
Finally RabbitMQ and Scrapyd ports are exposed for consumption

Name		Name	Last commit message	Last commit date
Latest commit History 28 Commits
gustoqa		gustoqa
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
dependencies.txt		dependencies.txt
docker-compose.yml		docker-compose.yml
packages.txt		packages.txt
requirements.txt		requirements.txt
scrapy.cfg		scrapy.cfg