Hey dude,
Welcome to your new web scraper!
-
Download the project as a zip file.
-
Unzip the file.
-
Open your terminal app, navigate to the folder you just unzipped, and while you are on the project folder type -
$ virtualenv env
if you get an error (which means that the Python virtualenv package is not installed on your machine), run -
$ pip install virtualenv
-
Open the Python virtual environment by typing -
$ source env/bin/activate
-
Install all project dependencies by typing -
(env)$ pip install -r requirements.txt
All good?
Let's rock 😎
-
You set up the list of websites you want to scrape on the
websites.txt
file. -
You set up the companies you want to want to look for on the
companies.txt
file. -
While inside the virtual environment and inside the project folder, run -
python scraper.py
The results should be:
-
A list of articles on the terminal.
-
A CSV file on the project folder.