This is a web content extracting module written in Python, it's heavily based on python lxml. It works best on Deep websites (websites that result information based on what you entered) like Amazon, StackOverflow, Ebay,..

Given a front page url, it will extract all products/articles links from it (including pagination). It can also extract users comment about the products/articles.

• Bottom-Up Region Extractor for Semi-Structured Web Pages - Wachirawut Thamviset, Sartra Wongthanavasu Pdf
• Demo video: youtube
• Extracting Informative Textual Parts from Web Pages Containing User-Generated Content: pdf