Skip to content

wenxinwilliam/docker-django-celery

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Dockerized Async Job Queue Websocket Feedback - Project Template / Demo - Work In Progress

Overview:
  1. User logs in to django app, then a one time token is generated and stored to redis cache and sent back to user
  2. User uses the token to authenticate and establish a websocket connection to aiohttp async webserver
  3. User schedules a job (poentially long running) for celery to process
  4. Celery done job processing then sends status update through rabbitmq to aiohttp async server
  5. Async server confirms logged in user and active websocket connection then sends message to the frontend
  6. User gets alert then page refresh
Running

Install docker, docker-compose and run docker-compose build (only need to build once unless Dockerfile changes) then docker-compose up (see notes below). Django is on port 8008 of docker and aiohttp async server is on port 8009 (refer to docker-compose.yml).

(Temporary steps - after running docker-compose build)

  1. Right now to start the system, use docker-compose up -d then do docker start dockerdjangocelery_worker_1 (because worker container bootstrapping with message queue needs to be fixed)
  2. Either install webpack (npm install -g webpack) on host or ssh into docker (docker exec -it dockerdjangocelery_app_1 bash) and run webpack --watch --colors inside mydjangoapp/mydjangoapp/static/apps/main/ to generate compiled static files (pending automation)
  3. Create admin user using docker exec -it dockerdjangocelery_app_1 python manage.py createsuperuser, then login at docker_hostname_or_ip:8008/admin/ (will add those pages)
  4. Go to docker_hostname_or_ip:8008/ to view jobs or add job

In case of any problem, use docker ps and check if the following containers are running:

  • dockerdjangocelery_asyncapp_1
  • dockerdjangocelery_worker_1
  • dockerdjangocelery_app_1
  • dockerdjangocelery_rabbitmq_1
  • dockerdjangocelery_redis_1
  • dockerdjangocelery_db_1

Check their logs using docker logs -f container_name

Main Stack

Backend

  • docker
  • django
  • aiohttp (async python)
  • redis
  • celery
  • rabbitmq
  • postgres

Frontend

  • webpack
  • backbone, marionette
  • bootstrap
  • ...
Disclaimer

This project is only for proof of concept, not optimized for production or security

Original project can be found here.

About

Dockerized Async Job Queue Websocket Feedback

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published