Proxypool based on Python3

README_CN.md in Chinese

conda install --yes --file requirements.txt

If you set up redis authentication, you need to set 'REDIS_PASSWORD' in settings.py

Proxypool provides several crawlers on free proxies. It can be seen in proxy_web.txt

If needed, you can add crawlers to crawler.py

cd into the project folder and input the command below:

python run.py

Input http://127.0.0.1:5555 in the browser, then you can see the page by WebAPI encapsulation

/random：provide a proxy randomly

/count：provide the total amount of proxypool

It is very convenient to get proxy by WebAPI.

import requests

PROXY_POOL_URL = 'http://localhost:5555/random'

def get_proxy():
    try:
        response = requests.get(PROXY_POOL_URL)
        if response.status_code == 200:
            return response.text
    except ConnectionError:
        return None

1. Project skeleton is based on Python3Webspider
2. Comments and suggestions are most welcome!