This is a web app showing latest data from mik4el's gadgets using Django, Angular2 and Docker. The app is currently available at https://m4bd.se.
The web app is separated into a backend web/gadget_board_backend
using Django and a frontend web/gadget_board_frontend
using Angular2.
The full stack is running in Docker containers and consists of:
- "nginx_dev" or "nginx_prod": nginx for web server and serving static files.
- "web": Django served by gunicorn.
- "postgres": the postgres database.
- Persisting data volumes "postgres_data_dev" or "postgres_data" for the database.
This is also my way to learn what can be considered to be efficient development and deployment workflows in 2016. My goal's for the workflow is:
- Closely mimick development environment and production environment
- Quick feedback loop from code change to feature in development
- Fast deployment to production of the entire stack as well as small updates
- Entire stack is in repo
- Easy to add services
- Deployment agnostic for hosting providers
Suggested reading and inspiration for this repo: https://realpython.com/blog/python/django-development-with-docker-compose-and-machine/ (Good concept and introduction but some code is out-of-date and following instructions will not give a working setup, see blog comments for more info)
- Docker-Toolbox with Docker-Compose >1.7.1 (https://www.docker.com/products/docker-toolbox)
- At least node v5.x.x and npm 3.x.x
- gulp installed globally (
sudo npm install --global gulp-cli
)
This will start a new development environment and serve the web app on your machine. This requires the download of all depedencies which will take some time.
docker-machine create -d virtualbox dev
eval $(docker-machine env dev)
- Make your own
.env
-file with credentials e.g.cp .env_template .env
docker-compose build
docker-compose up -d
docker-compose run --rm web python manage.py migrate
docker-compose run --rm web python manage.py createsuperuser
cd web
npm install
gulp build
- Open a browser at the ip from
docker-machine ip dev
For normal development work, I suggest this workflow:
- Make change in Django related code, run
manage.py
commands usingdocker-compose run --rm web python manage.py startapp
etc... - Reload browser, run tests etc.
Special cases:
- Make sure no old code is running if you are changing e.g.
settings.py
orurls.py
by restarting the web container. Do this withdocker-compose restart web
. - Run
manage.py
commands usingdocker-compose run --rm
, e.g.docker-compose run --rm web python manage.py makemigrations
. - If you restart your computer etc, you may need to restart the machine running the containers, do so by:
docker-machine start dev
eval $(docker-machine env dev)
(also when you open a new terminal)
When you add a dependency to web/requirements.txt
you need to build a new container image and restart the container with this new image, this is done by:
docker-compose build web
docker-compose up -d
- Possibly run
docker-compose run --rm web python manage.py collectstatic
or other commands your dependency requires. NB: collectstatic needs to be run in the development workflow for static files to be saved in the static-dir, this is not possible to do on deployed containers.
For normal development work, I suggest this workflow:
- Make change in source file in
gadget_board_frontend
gulp build
- Reload browser
Angular2 dependencies are handled by npm, built by gulp and loaded by systemjs, therefore:
- Add dependency in
package.json
. - If the dependency should be used in the app and served as a static file:
- Add path to node_module for dependency in gulpfile.js either as file or dir in tasks
copy:lib_dirs
orcopy:lib_files
- Add path to dependency in
systemjs.config.dev.js
and if necessary insystemjs.config.prod.js
npm install
gulp build
- Reload browser
Now we need to set up the production environment to which you are deploying. By using Docker the production environment is very agnostic to what provider you choose. I like DigitalOcean for small projects that can grow but there are many options. Doing the first deployment requires you to migrate the database for the first time, also we use a different docker-compose file so you need to rebuild the container images.
- Get working SSL certs e.g. by following the steps below.
- Get an access token for your DigitalOcean account.
docker-machine create -d digitalocean --digitalocean-access-token=<token> --digitalocean-region=fra1 production
(use same region where your floating ip is for the domain you use in SSL)eval $(docker-machine env production)
docker-compose -f production.yml build
docker-compose -f production.yml up -d
docker-compose -f production.yml run --rm web python manage.py migrate
Since the app handles user data securing traffic using SSL is a requirement for production. To set it up on the production nginx container I followed these steps:
- Have a domain ready that you control. It is nice to point the domain towards a floating ip from e.g. DigitalOcean so you can change the production environment without needing to update your dns.
- Find a suitable SSL certificate authority (CA), I use positivessl from namecheap.com. For SSL certs with short expiration dates there are also free options but that is a hassle so I value buying an SSL cert. A better free option is https://letsencrypt.org/.
- Make a csr-file by running
openssl req -newkey rsa:2048 -nodes -keyout example.com.key -out example.com.csr
and go through the process of obtaining the cert. Save the key and csr file safely on your local machine. - When you have got all the cert files back from your CA, prepare the cert for nginx by following e.g.
https://www.namecheap.com/support/knowledgebase/article.aspx/9419/0/nginx
. Store the files safely on your local machine. - Download root certificate from your CA, e.g.
https://www.namecheap.com/support/knowledgebase/article.aspx/9393/69/where-do-i-find-ssl-ca-bundle
, save it asPositiveSSLBundle.cer
- Make a dhparam.pem file by running
openssl dhparam 2048
and store safely. - Copy over the files to a new directory
ssl
in foldernginx
. They will not be added to git.
When you are deploying the next time we also need to rebuild the container that has changed files since the production environment does not mount your local machines files.
gulp build-prod
docker-compose -f production.yml build
docker-compose -f production.yml down
docker-compose -f production.yml up -d
NB: If in a new terminal remember eval $(docker-machine env production)
.
Testing in Django is handled by the default Django test system, so running tests is easy, e.g:
docker-compose run --rm web python manage.py test
Testing in Angular2 is handled by jasmine, run the tests by:
- Going to
/unit-tests.html
in your browser. - Add a test by following e.g. https://angular.io/docs/ts/latest/guide/testing.html and then import the spec-file in
unit-tests.html
A deployed environment can be backed up by your hosting provider, e.g. DigitalOcean. Since this is a very stateless deployment you can also make a scripted backup of your database and make it possible to easily restore the database from a backup. This will save some on hosting costs and make for a more self-contained and hosting provider agnostic solution.
- Stack not optimized for performance
- Non optimized frontend build system
- App and stack not thourougly tested
- Development and deploy workflow not thouroughly tested
- https://cli.angular.io/ is coming but not stable yet, this should be the favored build system for angular2
- Styling is limited to one app.css atm, should be modular and built along with e.g.
bootstrap
usingless
.
- gulpjs/gulp#1571
- Test for services broken after rc5, fix when Angular2 updates documentation
- Frontend: Animation when going to new view
- Frontend: Animation when data updated
- Frontend: Other component for other gadget parsing its gadget data
- Frontend: Use latest angular2-router
- Frontend: Redirect to login if unauthed when accessing protected view
- Refresh JWT tokens in background
- Fix space and tabs inconsistency in code.
- Neat css builds with less and bootstrap
- Suggest simple script for backup.
- Add more gadgets!
- Cleaner build system for Angular2
- Test https://www.docker.com/products/docker#/mac