Skip to content

jgoclawski/async-crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 

Repository files navigation

A Web Crawler With asyncio Coroutines

It's a git repo with code samples based on "Web crawler" chapter from "500 Lines or Less".

It uses an incremental approach, so you can follow along the chapter and run code samples with verbose logging.

Uses Docker and Python 3.6.

The code values output more than readability (you can get that in the book itself!). As Docker is the only dependency, it makes it easy to run (and change!) the code and observe the results.

Try it yourself, just have Docker installed and run:

# Run the first example - 01_async.py
./run.sh

# Run selected file - 04_refactored_coroutine_with_generators.py
./run.sh 04_refactored_coroutine_with_generators.py

Licenses

The written material in the book is licensed under Creative Commons Attribution and the code samples are licensed under the MIT license. The code in this repo is licensed under the MIT license as well.

About

A Web Crawler With asyncio Coroutines

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published