Skip to content

zhoushuqiang/PyCrawler

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Python Crawler(auto download from website)

Required

  • OS:windows(maybe Linux and mac)
  • Pyhon:v2.7.12+, not supported Python 3.X

Features

  • 多线程
    multithreading
  • 支持使用代理设置
    support proxy
  • 支持绑定键盘事件(快捷键),如暂停/启动程序运行
    support bind keyboard events, e.g. pause or restart process
  • 支持从浏览器中读取cookies并携带访问限制的网站
    read cookies from your browser
  • 页面访问支持多种参数:
    support multiple parameter for visit web
    • 支持常用请求方法:GET、POST、HEAD、PUT、DELETE、OPTIONS、TRACE
      common request method: GET, POST, HEAD, PUT, DELETE, OPTIONS, TRACE
    • 可自定义添加request header
      add customize request header
    • 可自定义添加Cookies
      add customize cookies
    • 可设置链接超时、读取超时
      set connection timeout and read timeout
    • 可设置是否自动跳转(http code 301、302、303、307、308)
      set whether auto redirect(http code 301, 302, 303, 307, 308)

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 99.9%
  • Batchfile 0.1%