Skip to content

Kryndex/scrapinghub-entrypoint-scrapy

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

scrapinghub-entrypoint-scrapy

Scrapy entrypoint for Scrapinghub job runner.

The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.

Features

  • parsing job data from environment
  • processing job args and settings
  • running a job with Scrapy
  • collecting stats
  • advanced logging & error handling
  • full hubstorage support
  • custom scripts support

Install

pip install scrapinghub-entrypoint-scrapy

About

Scrapy entrypoint for Scrapinghub job runner

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%