Python Crawler._scrape_pages示例

示例#1

文件： tests.py 项目： zamattiac/ROSIEBot

 def test_scrape_url(self):
     c = Crawler(d)
     try:
         c._scrape_pages(['http://google.com', 'http://google.com/'])
         f = open('google.com/index.html')
         f.close()
     except:
         self.fail("page didn't save / get scraped at all")

示例#2

 def test_scrape_url(self):
     c = Crawler(d)
     try:
         c._scrape_pages(['http://google.com', 'http://google.com/'])
         f = open('google.com/index.html')
         f.close()
     except:
         self.fail("page didn't save / get scraped at all")