Wextracto: Web Data Extraction Wextracto is a toolkit for command-line web data extraction. Installation $ pip install wextracto Kicking the Tyres $ echo -e "[wex]\nsitemaps=wex.sitemaps:urls_from_sitemaps" > entry_points.txt $ wex "http://www.ebay.com/robots.txt" Documentation The documentation can be found here: http://wextracto.readthedocs.org/en/latest/index.html