Skip to content

petakajaib/building-a-news-crawler

Repository files navigation

Building a news crawler

This is the accompanying code for Petak Ajaib

Once everything is installed, to run all:

./run_all.sh

We encourage you to read the code.

If you have to re-run the initial_crawl be sure remove the done_initial_crawl file. We check the existance of this file inside run_all.sh to know whether we have done the initial crawl or not.

rm done_initial_crawl