Skip to content

A suite for scraping site news and generate a bag of words analysis. Then the system wil scrape social networks and analyze the impact of every news.

License

Notifications You must be signed in to change notification settings

krukmat/smartnews

Repository files navigation

Smartnews: The highway to hell for news

Web platform which allows the users to 'scrape' from their favorite news site and have a refined landscape for everyday hot topics.

How to use it locally? (Linux - Mac)

  • git clone or easily download project in Zip format.

Server:

  • Install vagrant
  • In project's folder:
  1. sudo vagrant init.
  2. sudo fab vagrant install.

Then you can call the task that creates the topics:

from scraper_module.tasks import scrape_news
scrape_news.delay()

The web module is still on development. But idea is user can signup, select the scrapers he wants and the user will have the news in a tag cloud format with all the sites related to every topic

ROAD MAP:

  • 0.1: Simple version with everyday hot-topic
  • 0.2: Multiuser support. User can select in a list of predefined scrapers.
  • 0.3: User can create 'alarms' looking for some concepts or topics.
  • 0.4: User can create its own scrapers.
  • 0.5: Integration with social networks.
  • 0.6: Integration with blog services (widget creation)
  • 0.7: Migration to AngularJS. API Generation

About

A suite for scraping site news and generate a bag of words analysis. Then the system wil scrape social networks and analyze the impact of every news.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published