Python add_downloadmw_to_crawler Examples

Programming Language: Python

Namespace/Package Name: scrapingtools.lib.utils

Method/Function: add_downloadmw_to_crawler

Examples at hotexamples.com: 2

Python add_downloadmw_to_crawler - 2 examples found. These are the top rated real world Python examples of scrapingtools.lib.utils.add_downloadmw_to_crawler extracted from open source projects. You can rate examples to help us improve the quality of examples.

Example #1

Show file

File: Proxy.py Project: Xingzhouliu1987/scrapingtools

	def from_crawler(cls, crawler):	  
		obj = cls(crawler.settings, crawler.stats, crawler)
		crawler.signals.connect(obj.spider_opened, signals.spider_opened)
		crawler.signals.connect(obj.spider_closed, signals.spider_closed)
		crawler.signals.connect(obj.proxies.spider_opened, signals.spider_opened)	
		crawler.signals.connect(obj.proxies.spider_closed, signals.spider_closed)	
		crawler.proxies = obj
		return add_downloadmw_to_crawler(obj,crawler)

Example #2

Show file

File: preventdupesmiddleware.py Project: Xingzhouliu1987/scrapingtools

	def from_crawler(cls, crawler):
		unique_index = cls(crawler.settings, crawler)
		crawler.signals.connect(unique_index.spider_opening, signal=signals.spider_opened)
		crawler.signals.connect(unique_index.spider_closing, signal=signals.spider_closed)
		crawler.unique_index = unique_index
		return add_downloadmw_to_crawler(unique_index,crawler)