Python OverusedBufferScrapy Examples

Example #1

File: backend.py Project: hoardboard/memex-pinterest

class MongodbScoreBackendOverused(MongodbScoreBackend):
    def __init__(self, manager):
        super(MongodbScoreBackendOverused, self).__init__(manager)
        self._requests_buffer = OverusedBufferScrapy(super(MongodbScoreBackendOverused, self).get_next_requests,
                                               manager.logger.manager.debug)
    def get_next_requests(self, max_n_requests, **kwargs):
        return self._requests_buffer.get_next_requests(max_n_requests, **kwargs)

Example #2

File: backend.py Project: hoardboard/memex-pinterest

 def __init__(self, manager):
     super(MongodbScoreBackendOverused, self).__init__(manager)
     self._requests_buffer = OverusedBufferScrapy(super(MongodbScoreBackendOverused, self).get_next_requests,
                                            manager.logger.manager.debug)