Author

Stefan Eletzhofer

Date

2011-10-19

Version

0.1

The goal of this project is to test out a plan to move from pika to kombu in the nexiles.settr.export project.

The system designes is a very simple job processing system, where a external tool/server initiates some job and this system needs to:

• process the initial job and gather information regarding the job
• decompose the initial job to smaller pieces
• send those smaller pieces to one dedicated entity in the system which processes these pieces. These job processors shall be created on-the-fly.

R001 Job Queues Purgeable

The administrator shall be enabled to use the RabbitMQ admin gui to purge the job queues. The system shall handle this gracefully.

We have some conditions for which we need to design as follows:

• The initial job needs to be decomposed into smaller pieces
• The set of jobs created from the decomposition need to be handled by a separate entity as the job processing itself is very instable

We'll have the following components:

master

Responsible to listen for new job announcements. Starts a new slave for each announcement which will handle the jobs.

slave

The slave which will handle the jobs coming in. The slave will listen on its own queue.

kombu docs

http://ask.github.com/kombu/index.html

kombu nexiles fork

git@github.com:nexiles/kombu.git