Skip to content

A web scraper that gets news articles about a list of companies from a list of websites

Notifications You must be signed in to change notification settings

dodiku/news_scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Fernando's news scraper

Hey dude,
Welcome to your new web scraper!

scraper

How to install?

  1. Download the project as a zip file.

  2. Unzip the file.

  3. Open your terminal app, navigate to the folder you just unzipped, and while you are on the project folder type -
    $ virtualenv env
    if you get an error (which means that the Python virtualenv package is not installed on your machine), run -
    $ pip install virtualenv

  4. Open the Python virtual environment by typing -
    $ source env/bin/activate

  5. Install all project dependencies by typing -
    (env)$ pip install -r requirements.txt

All good?
Let's rock 😎

How to use?

  1. You set up the list of websites you want to scrape on the websites.txt file.

  2. You set up the companies you want to want to look for on the companies.txt file.

  3. While inside the virtual environment and inside the project folder, run - python scraper.py

Results

The results should be:

  • A list of articles on the terminal.

  • A CSV file on the project folder.

About

A web scraper that gets news articles about a list of companies from a list of websites

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages