Ejemplos de get_untouched_question_cursor en Python

Ejemplo n.º 1

Archivo: main.py Proyecto: cznyx/zhihu-spider

    def fetch_questions(self):
        questions_list_cursor = get_untouched_question_cursor(self.db, EXPLORE_QUESTION_COUNT)
        for question_item in questions_list_cursor:
            question = question_item['question_id']
            self.update_question_insert_answer(question)

        log.info("Finished crawling {} questions.".format(EXPLORE_QUESTION_COUNT))

Ejemplo n.º 2

Archivo: main.py Proyecto: ahxxm/zhihu-spider

    def fetch_questions(self):
        questions_list_cursor = get_untouched_question_cursor(self.db, EXPLORE_QUESTION_COUNT)
        question_list = [question_item["question_id"] for question_item in questions_list_cursor]
        tasks = asyncio.wait([self.update_question_insert_answer(question_id)
                              for question_id in question_list])
        self.loop.run_until_complete(tasks)

        log.info("Finished crawling {} questions.".format(EXPLORE_QUESTION_COUNT))