If you are a nicecream.fm addict, I got you covered. You can now have all those songs playing each and every minute, even when you are not online 😁.
The web app is made of an API and a SPA.
Features:
- Channels history
- Social sign in with Google
- Bookmarks
- ...
References:
- OAuth2 Implicit Grant and SPA
- Using OAuth 2.0 for Web Server Applications
- Swagger Cookie Authentication
Requirements:
- Docker
- Docker Compose
- GPC account
- Google OAuth2 client with
http://api.lvh.me:8080/user/google/callback
added to Authorized redirect URIs
Build the api
image:
docker-compose build api
Generate a Fernet secret key that will be used to encrypt the session cookie:
docker run --rm nicecream_history_api python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
Copy .env.sample
to .env
and fill in API_SESSION_COOKIE_SECRET_KEY
, API_GOOGLE_CLIENT_ID
, API_GOOGLE_CLIENT_SECRET
:
cp .env.sample .env
Run and open http://api.lvh.me:8080
in browser:
docker-compose up -d
References:
Describe and provision infrastructure with CloudFormation.
Requirements:
- Internet domain (i.e example.com) and access to DNS management 😎
- Docker
- Docker Compose
- GPC account
- Google OAuth2 client with
https://api.[example.com]/user/google/callback
added to Authorized redirect URIs - AWS account
- AWS CLI
- AWS certificate for
*.example.com
generated into the region where deployment will be done.
References:
The bucket will hold all CloudFormation templates.
aws cloudformation create-stack \
--stack-name [CLOUDFORMATION_STACK_NAME] \
--template-body file://cloudformation/templates/s3.yaml \
--parameters \
ParameterKey=Access,ParameterValue=PublicRead \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME] \
--capabilities CAPABILITY_IAM
Upload:
aws s3 sync ./cloudformation/templates s3://[CLOUDFORMATION_BUCKET]
Receive alerts from resources. Supported transports are: Slack, Email, HTTP endpoint, HTTPS endpoint
aws cloudformation create-stack \
--stack-name [ALERT_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/alert.yaml \
--parameters \
ParameterKey=FallbackEmail,ParameterValue=[FALLBACK_EMAIL] \
ParameterKey=Email,ParameterValue=[EMAIL] \
--capabilities CAPABILITY_IAM
./slack_alert_lambda.sh [SLACK_LAMBDA_FUNCTION_NAME]
VPC with 2 public & 2 private subnets in two availability zones (Zone A & B).
aws cloudformation create-stack \
--stack-name [VPC_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/vpc-2azs.yaml \
--parameters \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME]
Security group used for gaining access to resources which are not integrated into the VPC (i.e ElastiCache).
aws cloudformation create-stack \
--stack-name [CLIENT_SG_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/client-sg.yaml \
--parameters \
ParameterKey=ParentVPCStack,ParameterValue=[VPC_STACK_NAME] \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME]
EC2 instance used for NAT and as a SSH Bastion (optional).
Best practices recommends to have one independent stack for each availability zone.
NAT (per availability zone):
Must deploy for each zone A & B.
aws cloudformation create-stack \
--stack-name [VPC_NAT_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/vpc-nat-instance.yaml \
--parameters \
ParameterKey=ParentVPCStack,ParameterValue=[VPC_STACK_NAME] \
ParameterKey=ParentAlertStack,ParameterValue=[ALERT_STACK_NAME] \
ParameterKey=SubnetZone,ParameterValue=[SUBNET_ZONE] \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME] \
--capabilities CAPABILITY_IAM
NAT (per VPC):
aws cloudformation create-stack \
--stack-name [VPC_NAT_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/vpc-nat-instance.yaml \
--parameters \
ParameterKey=ParentVPCStack,ParameterValue=[VPC_STACK_NAME] \
ParameterKey=ParentAlertStack,ParameterValue=[ALERT_STACK_NAME] \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME] \
--capabilities CAPABILITY_IAM
NAT and SSH (per VPC):
aws cloudformation create-stack \
--stack-name [VPC_NAT_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/vpc-nat-instance.yaml \
--parameters \
ParameterKey=ParentVPCStack,ParameterValue=[VPC_STACK_NAME] \
ParameterKey=ParentAlertStack,ParameterValue=[ALERT_STACK_NAME] \
ParameterKey=ParentClientStack,ParameterValue=[CLIENT_SG_STACK_NAME] \
ParameterKey=KeyName,ParameterValue=[EC2_KEY_NAME] \
ParameterKey=SSHAccessCidrIp,ParameterValue='[IPv4_SSH_ALLOWED]/32' \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME] \
--capabilities CAPABILITY_IAM
Multi-AZ/Replica disabled by default.
aws cloudformation create-stack \
--stack-name [RDS_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/rds-postgres.yaml \
--parameters \
ParameterKey=ParentVPCStack,ParameterValue=[VPC_STACK_NAME] \
ParameterKey=ParentClientStack,ParameterValue=[CLIENT_SG_STACK_NAME] \
ParameterKey=ParentAlertStack,ParameterValue=[ALERT_STACK_NAME] \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME] \
ParameterKey=DBMasterUsername,ParameterValue=[DATABASE_USERNAME] \
ParameterKey=DBMasterUserPassword,ParameterValue=[DATABASE_PASSWORD] \
ParameterKey=DBName,ParameterValue=[DATABASE_NAME]
Multi-AZ/Replica disabled by default.
aws cloudformation create-stack \
--stack-name [ELASTICACHE_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/elasticache-redis.yaml \
--parameters \
ParameterKey=ParentVPCStack,ParameterValue=[VPC_STACK_NAME] \
ParameterKey=ParentClientStack,ParameterValue=[CLIENT_SG_STACK_NAME] \
ParameterKey=ParentAlertStack,ParameterValue=[ALERT_STACK_NAME] \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME]
aws cloudformation create-stack \
--stack-name [S3_ACCESS_LOG_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/s3.yaml \
--parameters \
ParameterKey=Access,ParameterValue=ElbAccessLogWrite \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME] \
--capabilities CAPABILITY_IAM
ECS Cluster with one public ALB.
Cluster autoscaling based on A better solution to ECS AutoScaling. Things to consider:
- maximum memory (default=256MiB) and maximum CPU (default=256Units)
- container shortage (default=0) and excess (default=4) thresholds
- instance type (default=t2.micro)
aws cloudformation create-stack \
--stack-name [ECS_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/ecs-cluster.yaml \
--parameters \
ParameterKey=ParentVPCStack,ParameterValue=[VPC_STACK_NAME] \
ParameterKey=ParentAlertStack,ParameterValue=[ALERT_STACK_NAME] \
ParameterKey=ParentClientStackOne,ParameterValue=[CLIENT_SG_STACK_NAME] \
ParameterKey=ParentS3StackAccessLog,ParameterValue=[S3_ACCESS_LOG_STACK_NAME] \
ParameterKey=SubnetsReach,ParameterValue=Private \
ParameterKey=SSMParametersPrefix,ParameterValue=\"[AWS_SSM_PREFIX]\" \
ParameterKey=LoadBalancerCertificateArn,ParameterValue=[SSL_CERTIFICATE_ARN] \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME] \
--capabilities CAPABILITY_IAM
Add a CNAME record for api.example.com
, pointing to ECS cluster ALB:
api.example.com. 3600 IN CNAME XXX.YYY.elb.amazonaws.com.
Encryption key used on SSM parameters encryption.
aws cloudformation create-stack \
--stack-name [KMS_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/kms.yaml \
--parameters \
ParameterKey=ParentVPCStack,ParameterValue=[VPC_STACK_NAME] \
ParameterKey=ParentClusterStack,ParameterValue=[ECS_STACK_NAME] \
ParameterKey=Administrators,ParameterValue=\"$(aws iam get-group --group-name [IAM_GROUP] --output text | awk 'BEGIN {FS="\t";ORS=","} {print $2}' | sed 's/,$//')\" \
ParameterKey=EnvironmentName,ParameterValue=[ENVIRONMENT_NAME]
Check .env.sample
and api/settings.py
for needed parameters.
Required parameters for production:
- PGHOST (String)
- PGPORT (String)
- PGUSER (String)
- PGPASSWORD (SecureString)
- PGDATABASE (String)
- REDIS_HOST (String)
- REDIS_PORT (String)
- SPA_URL (String)
- API_SESSION_COOKIE_SECRET_KEY (SecureString)
- API_SESSION_COOKIE_DOMAIN (String)
- API_GOOGLE_CLIENT_ID (String)
- API_GOOGLE_CLIENT_SECRET (SecureString)
- API_GOOGLE_REDIRECT_URL (String)
- API_CSRF_COOKIE_DOMAIN (String)
- API_CORS_ALLOWED (String)
- API_CORS_ORIGIN (String)
aws ssm put-parameter \
--name "/[AWS_SSM_PREFIX]/[PARAMETER_NAME]" \
--type "String" \
--value "[PARAMETER_VALUE]"
aws ssm put-parameter \
--name "/[AWS_SSM_PREFIX]/[PARAMETER_NAME]" \
--type "SecureString" \
--value "[PARAMETER_VALUE]" \
--key-id [KMS_KEY_ID]
Create an ECR repository and upload the API image before starting ECS services.
$(aws ecr get-login --no-include-email --region [AWS_REGION])
docker build \
--tag [REPOSITORY_NAME]:latest \
--tag [REPOSITORY_NAME]:$(echo $(git rev-parse --short HEAD)) .
docker push [REPOSITORY_NAME]:latest && \
docker push [REPOSITORY_NAME]:$(echo $(git rev-parse --short HEAD))
Migrations task definition. Uses the same API Docker image.
aws cloudformation create-stack \
--stack-name [MIGRATIONS_TASK_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/ecs-task.yaml \
--parameters \
ParameterKey=ParentClusterStack,ParameterValue=[ECS_STACK_NAME] \
ParameterKey=Image,ParameterValue=[REPOSITORY_NAME]:latest \
ParameterKey=SSMParametersPrefix,ParameterValue=\"[AWS_SSM_PREFIX]\" \
ParameterKey=Command,ParameterValue=\"alembic,upgrade,head\"
aws ecs run-task \
--cluster [ECS_CLUSTER_NAME] \
--task-definition [MIGRATIONS_TASK_STACK_NAME]
Uses the same API Docker image.
Default maximum memory = 256MiB and default maximum CPU = 256Units.
Review ECS Cluster autoscaling on maximum memory & CPU changes.
aws cloudformation create-stack \
--stack-name [CRAWLER_SERVICE_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/ecs-service-worker.yaml \
--parameters \
ParameterKey=ParentClusterStack,ParameterValue=[ECS_STACK_NAME] \
ParameterKey=ParentAlertStack,ParameterValue=[ALERT_STACK_NAME] \
ParameterKey=DesiredCount,ParameterValue=1 \
ParameterKey=MinCapacity,ParameterValue=1 \
ParameterKey=MaxCapacity,ParameterValue=1 \
ParameterKey=Image,ParameterValue=[REPOSITORY_NAME]:latest \
ParameterKey=SSMParametersPrefix,ParameterValue=\"[AWS_SSM_PREFIX]\" \
ParameterKey=Command,ParameterValue=\"python,-m,crawler\" \
--capabilities CAPABILITY_IAM
Default maximum memory = 256MiB and default maximum CPU = 256Units.
Review ECS Cluster autoscaling on maximum memory & CPU changes.
aws cloudformation create-stack \
--stack-name [API_SERVICE_STACK_NAME] \
--template-url https://s3.amazonaws.com/[CLOUDFORMATION_BUCKET]/ecs-service.yaml \
--parameters \
ParameterKey=ParentVPCStack,ParameterValue=[VPC_STACK_NAME] \
ParameterKey=ParentClusterStack,ParameterValue=[ECS_STACK_NAME] \
ParameterKey=ParentAlertStack,ParameterValue=[ALERT_STACK_NAME] \
ParameterKey=Image,ParameterValue=[REPOSITORY_NAME]:latest \
ParameterKey=SSMParametersPrefix,ParameterValue=\"[AWS_SSM_PREFIX]\" \
ParameterKey=HealthCheckPath,ParameterValue=channels \
ParameterKey=LoadBalancerHttps,ParameterValue=true \
--capabilities CAPABILITY_IAM
Update:
aws ecs update-service \
--cluster [ECS_CLUSTER_NAME] \
--force-new-deployment \
--service [ECS_SERVICE_NAME]