Skip to content

mjstallard/bernie_scrapers

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Bernie Scrapers

This repository contains the scrapers used by ES4BS to import events and articles (and any other additional data) into the Mongo DB source of truth.

Deploy

Scrapers are triggered by scraper_scheduler using docker to isolate the code. All scrapers should utilize the Scraper base class, and will be executed 2 at a time every 20 minutes. Dead scrapers are left for 24 hours for logging purposes.

About

Scrapers for pulling information into ES4BS

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%