Skip to content

ikrestov/rq-gevent-worker

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

34 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

rq-gevent-worker

Implement a new worker based on gevent

Downloads

##Install this fork

pip install --upgrade -e git+https://github.com/relic-/rq-gevent-worker.git@master#egg=rq-gevent-worker

##Install

$ pip install rq-gevent-worker

##Usage

$ rqgeventworker -h

$ export PYTHONPATH=<your project import path>:$PYTHONPATH; rqgeventworker

##Test

$ pip install -r requirements.txt

$ py.test tests

##Under The Hood TODO

##TODO

  • Add a command line option to specify gevent pool size

##Note

###Crash Official Worker use os.fork() to spawn a child process to execute a job, so if the job cause the process crash, the worker process is still alive.

When using gevent, we use the same process to execute job, the job may cause the whole worker process crash.

###Why not rqworker -w <geventworker> Because we need gevent monkey patch at the start of the process, rqworker import many modules before importing geventworker, so it will cause geventworker not work normally.

##Declaration

Most of the code is from lechup and jhorman,

About

A worker using gevent for rq

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%