newspaper is a python based article extraction framework. It can produce pocket-like results. While it is very easy to use as a python library, I thought it would be interesting to turn this into a hosted app. Its all setup to be heroku app.
git clone https://github.com/voidfiles/newspaper-delivery.git
cd newspaper-delivery
virtualenv --no-site-packages venv
source venv/bin/activate
pip install -r requirements.txt
python app.pyYou should now have a local version of the app running at http://127.0.0.1:5000/
You can now try it out there are two main endpoints. First is the /article endpoint. This works with newspapers article extraction.
curl http://localhost:5000/article?url=http://www.macdrifter.com/2014/01/list-of-natural-language-apis-link.html
Will produce something like this
{
"additional_data": {},
"authors": [],
"canonical_link": "http://www.macdrifter.com/2014/01/list-of-natural-language-apis-link.html",
"images": [
"http://www.macdrifter.com/theme/images/logo.svg",
...
],
"keywords": [
"rundown",
...
],
"meta_description": "",
"meta_favicon": "/theme/images/touch-icon-iphone.png",
"meta_keywords": [
""
],
"meta_lang": "en",
"movies": [],
"summary": [],
"tags": [
"Programming",
"Link"
],
"text": "This is a nice rundown of some natural language APIs.\n\nThe source is Mashape which looks like an API middleman. It's an interesting idea and their index looks very good and it's searchable.\n\nDon't miss the link at the bottom for PublicAPI.com, which I've seen before but always forget about.",
"title": "List of Natural Language APIs [Link]",
"top_image": "/theme/images/favicon.png",
"url": "http://www.macdrifter.com/2014/01/list-of-natural-language-apis-link.html"
}Lastly is the /source endpoint. Which is synonmous with the main interface of newspaper.
curl http://localhost:5000/source?url=http://cnn.com
Will produce something like this
{
"articles": [
"http://www.cnn.com/2012/04/04/opinion/colb-strip-search/index.html",
"http://ac360.blogs.cnn.com/2013/12/03/pick-your-favorite-ridiculist-of-2013/",
"http://piersmorgan.blogs.cnn.com/2013/11/08/william-shatner-on-obamacare-ponder-the-mystery/",
...
],
"brand": "cnn",
"categories": [
"http://cnn.com",
"http://cnn.com/HLN",
"https://portfolio.money.cnn.com",
"http://ireport.cnn.com",
...
],
"description": "CNN.com delivers the latest breaking news and information on the latest top stories, weather, business, entertainment, politics, and more. For in-depth coverage, CNN.com provides special reports, video, audio, photo galleries, and interactive guides.",
"domain": "cnn.com",
"favicon": "",
"feeds": [
"http://rss.cnn.com/rss/cnn_tech.rss",
...
],
"logo_url": ""
}