Skip to content
/ scaling-aws-ecs Public

Learn how to build and deploy a fault tolerant, scalable and load balanced AP on AWS ECS.

6 stars 4 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

smalltide/scaling-aws-ecs

BranchesTags

Repository files navigation

scaling-aws-ecs

Learn how to build and deploy a fault tolerant, scalable and load balanced AP on AWS ECS.

Scaling Docker on AWS
https://www.udemy.com/scaling-docker-on-aws

Skills

  1. Docker
  2. AWS
  3. AWS ECR
  4. AWS ECS
  5. AWS RDS
  6. AWS S3
  7. AWS ELB
  8. AWS ElastiCache

AWS Scaling

alt text

AWS ECS Compare with Other Services

alt text alt text alt text

Install docker, docker-machine, docker-compose on Linux

  > https://store.docker.com
  > https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/
  > curl -sSL https://get.docker.com/ | sh (install latest docker using script)
  > docker version
  > sudo usermod -aG docker <linux user> (add user in docker group)
  > curl -L https://github.com/docker/machine/releases/download/v0.12.2/docker-machine-`uname -s`-`uname -m` >/tmp/docker-machine && chmod +x /tmp/docker-machine && sudo cp /tmp/docker-machine /usr/local/bin/docker-machine (install docker machine)
  > docker-machine version
  > sudo -i
  > sudo curl -L 
  https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose (install docker compose)
  > chmod +x /usr/local/bin/docker-compose
  > exit
  > docker-compose version

AWS Set up Resource

Installing and Configuring the AWS CLI

  > sudo apt-get install curl
  > curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip" 
  > unzip awscli-bundle.zip
  > sudo ./awscli-bundle/install -i /usr/local/aws -b / /usr/local/bin/aws
  > rm -rf awscli-bundle.zip awscli-bundle
  > aws --version
  > aws configure
  > aws iam list-users

Creating an SSH Keypair (local pc ssh can connect ec2 instance)

  > aws ec2 create-key-pair --key-name aws-ice --query 'KeyMaterial' --output text > ~/.ssh/aws-ice.pem
  > chmod 400 ~/.ssh/aws-ice.pem
  > aws ec2 describe-key-pairs
  > aws ec2 describe-key-pairs --key-names aws-ice
  > aws ec2 delete-key-pair --key-names aws-ice (for delete)

Creating a Security Group (Security Group used for setting ec2 network rule)

  > aws ec2 create-security-group --group-name ice_sg_ap-northeast-1 --description "security group for ice on ap-northeast-1"
  > aws ec2 describe-security-groups --group-id sg-cxxxxxxx
  > aws ec2 authorize-security-group-ingress --group-id sg-cxxxxxxx --protocol tcp --port 22 --cidr 0.0.0.0/0
  > aws ec2 authorize-security-group-ingress --group-id sg-cxxxxxxx --protocol tcp --port 80 --cidr 0.0.0.0/0
  > aws ec2 authorize-security-group-ingress --group-id sg-cxxxxxxx --protocol tcp --port 5432 --cidr 0.0.0.0/0 --source-group sg-cxxxxxxx (for RDS)
  > aws ec2 authorize-security-group-ingress --group-id sg-cxxxxxxx --protocol tcp --port 6379 --cidr 0.0.0.0/0 --source-group sg-cxxxxxxx (for redis)
  > aws ec2 describe-security-groups --group-id sg-cxxxxxxx
  > aws ec2 delete-security-group --group-id sg-cxxxxxxx

Creating ECS IAM Roles (used for manage aws resource access permission)

ecsInstanceRole alt text ecsServiceRole alt text

ECS Components

alt text

ECS Components Resource

AWS ECS Clusters

  > aws ecs create-cluster --cluster-name deepdive
  > aws ecs list-clusters
  > aws ecs describe-clusters --clusters deepdive
  > aws ecs delete-cluster --cluster deepdive
  > aws ecs create-cluster --cluster-name deepdive

alt text

ECS Container Agent

  > aws s3api create-bucket --bucket ecs-deepdive --region ap-northeast-1 --create-bucket-configuration LocationConstraint=ap-northeast-1
  > cd deepdive
  > aws s3 cp ecs.config s3:https://ecs-deepdive/ecs.config (upload)
  > aws s3 ls s3:https://ecs-deepdive
  > aws s3 cp s3:https://ecs-deepdive/ecs.config ecs.config (download)

alt text

Container Instances

  > cd deepdive
  > aws ec2 run-instances --image-id ami-21815747 --count 1 --instance-type t2.micro --iam-instance-profile Name=ecsInstanceRole --key-name aws-ice --security-group-ids sg-xxxxxx --user-data file:https://copy-ecs-config-to-s3 (aws-ice mean ssh key)
  > aws ec2 describe-instance-status --instance-ids i-06e2c776d598xxxxx
  > aws ec2 get-console-output --instance-id i-06e2c776d598xxxxx
  > aws ecs list-container-instances --cluster deepdive
  > aws ecs describe-container-instances --cluster deepdive --container-instances arn:aws:ecs:ap-northeast-1:28xxxxxxx:container-instance/01cf8a5e-6952-475c-b471-xxxxxxxx
  > aws ec2 terminate-instances --instance-ids i-06e2c776d598xxxxx (for delete instance)
  > ssh -i "aws-ice.pem" ec2-user@<ec2 instance ip> (ssh login ec2 instance)

alt text

Task Definitions

  > cd deepdive
  > aws ecs register-task-definition --cli-input-json file:https://web-task-definition.json
  > aws ecs list-task-definitions
  > aws ecs list-task-definition-families
  > aws ecs describe-task-definition --task-definition web:1
  > aws ecs register-task-definition --cli-input-json file:https://web-task-definition.json (see "revision": 1 to "revision": 2)
  > aws ecs list-task-definitions
  > aws ecs deregister-task-definition --task-definition web:2 (delete revision 2)
  > aws ecs list-task-definitions
  > aws ecs register-task-definition help
  > aws ecs register-task-definition --generate-cli-skeleton

alt text alt text

ECS Scheduler
alt text alt text alt text alt text alt text

Scheduling Services

  > cd deepdive
  > aws ecs create-service --cluster deepdive --service-name web --task-definition web --desired-count 1 (create 1 web service)
  > aws ecs list-services --cluster deepdive
  > aws ecs describe-services --cluster deepdive --services web
  > aws ec2 describe-instances (find Public DNS)
  > aws ecs update-service --cluster deepdive --service web --task-definition web --desired-count 2 (update web service to 2 instance)
  > aws ecs describe-services --cluster deepdive --services web (see only 1 runningCount, because the port 80 in used)
  > aws ecs update-service --cluster deepdive --service web --task-definition web --desired-count 0 (remove all task)
  > aws ecs delete-service --cluster deepdive --service web (delete web service)
  > aws ecs list-services --cluster deepdive
  > aws ecs create-service --generate-cli-skeleton

Run Task and Start Task

  > cd deepdive
  > aws ecs run-task --cluster deepdive --task-definition web --count 1 (auto select container-instance)
  > aws ecs list-tasks --cluster deepdive
  > aws ecs stop-task --cluster deepdive --task arn:aws:ecs:ap-northeast-1:2829XXXXXX:task/38890c8d-bd81-4998-be6d-527bXXXXXX
  > aws ecs list-tasks --cluster deepdive
  > aws ecs 
  > aws ecs list-container-instances --cluster deepdive
  > aws ecs start-task --cluster deepdive --task-definition web --container-instances arn:aws:ecs:ap-northeast-1:2829XXXXXX:container-instance/3233fc0a-8ba6-4698-abb4-817XXXXXX (specific select container-instance)
  > aws ecs stop-task --cluster deepdive --task arn:aws:ecs:ap-northeast-1:2829XXXXXX:task/b5d42a83-4276-46df-9cdb-0df6dXXXXXX

Private Docker Registry (ECR)

  > cd deepdive
  > aws ecr get-login --no-include-email --region ap-northeast-1 (see docker login -u AWS -p ......)
  > aws ecr create-repository --repository-name deepdive/nginx
  > aws ecr describe-repositories
  > aws ecr list-images --repository-name deepdive/nginx
  > docker pull nginx:1.9
  > docker image ls
  > docker tag nginx:1.9 2829XXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/deepdive/nginx:1.9
  > docker image ls
  > docker push 2829XXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/deepdive/nginx:1.9
  > aws ecr list-images --repository-name deepdive/nginx
  > aws ecs register-task-definition --cli-input-json file:https://web-task-definition.json
  > aws ecs run-task --cluster deepdive --task-definition web --count 1

alt text

ECS CLI

  > https://github.com/aws/amazon­ecs­cli
  > https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI_tutorial.html
  > ecs-cli images
  > ecs-cli ps --cluster deepdive

alt text

Tearing down Our Cluster

  > aws ec2 terminate-instances --instance-ids i-06e2c776d59XXXXXX
  > aws s3 rm s3:https://ecs-deepdive --recursive
  > aws s3api delete-bucket --bucket deepdive
  > aws s3api delete-bucket --bucket ecs-deepdive
  > aws ecr delete-repository --repository-name deepdive/nginx --force
  > aws ecs delete-cluster --cluster deepdive
  > aws ecs deregister-task-definition --task-definition web

Developing the Ruby on Rails Application

Generating a New Rails Project

  > cd scaling-aws-ecs
  > docker run -v "$PWD":/usr/src/app -w /usr/src/app rails:4 rails new --skip-bundle dockerzon
  > docker image ls
  > docker image rm rails (if want to delete rails image)

Running the Application Locally

  > cd dockerzon
  > docker-compose up
  > docker volume ls
  > docker network ls
  > docker container ls
  > docker exec dockerzon_dockerzon_1 rake db:reset
  > curl 127.0.0.1:8000
  > docker exec dockerzon_dockerzon_1 rake db:migrate
  > curl 127.0.0.1:8000

Working with the Application

  > cd dockerzon
  > docker-compose up
  > docker exec dockerzon_dockerzon_1 rails g model Dummy foo
  > docker exec dockerzon_dockerzon_1 rails d model Dummy
  > docker exec -it dockerzon_dockerzon_1 bash
  > ls -al
  > exit
  > docker exec -it dockerzon_dockerzon_1 rails c
  > exit

Building the Demo Application

  > cd dockerzon
  > docker-compose up
  > docker exec dockerzon_dockerzon_1 rake db:migrate
  > docker exec -it dockerzon_redis_1 redis-cli
  > KEYS *
  > GET dockerzon::cache:total_hits
  > exit
  > docker exec -it dockerzon_dockerzon_1 rails c
  > Javelin.all
  > Javelin.sum(:thrown)
  > Javelin.count
  > Javelin.all.pluck(:thrown)
  > exit
  > docker-compose down -v

Preparing to Deploy Everything on AWS

Using and Configuring nginx

  > cd nginx (revise PLACEHOLDER_VHOST in docker-entrypoint.sh)
  > docker build -t dockerzon_nginx .
  > cd dockerzon
  > docker-compose up -d (ROR APP run on 8000 port)
  > docker container ls -a
  > docker run --rm -p 80:80 --net dockerzon_default dockerzon_nginx (nginx proxy 80 to 8000 port)
  > curl 127.0.0.1:80
  > cd nginx (revise PLACEHOLDER_VHOST in docker-entrypoint.sh)
  > docker build -t dockerzon_nginx .
  > cd dockerzon 
  > docker-compose stop

alt text

Setting up an S3 Bucket

  > aws s3api create-bucket --bucket ecs-dockerzon --region ap-northeast-1 --create-bucket-configuration LocationConstraint=ap-northeast-1
  > aws s3 ls s3:https://ecs-dockerzon
  > cd s3-production-ecsconfig
  > aws s3 cp ecs.config s3:https://ecs-dockerzon/ecs.config (upload)
  > aws s3 ls s3:https://ecs-dockerzon

Setting up RDS for Postgres

  > aws rds create-db-instance --engine postgres --no-multi-az --no-publicly-accessible --vpc-security-group-ids sg-c782XXXX --db-instance-class db.t2.micro --allocated-storage 20 --db-instance-identifier dockerzon-production --db-name dockerzon_production --master-username dockerzon --master-user-password XXXXXXXX
  > aws rds modify-db-instance --db-instance-identifier dockerzon-production --master-user-password XXXXXX (if forget password for revise) 
  > aws rds describe-db-instances
  > aws rds delete-db-instance --db-instance-identifier docker-production --skip-final-snapshot (if want to delete)

alt text

Setting up ElastiCache for Redis

  > aws elasticache create-cache-cluster --engine redis --security-group-ids sg-c78XXXXX --cache-node-type cache.t2.micro --num-cache-nodes 1 --cache-cluster-id dockerzon-production
  > aws elasticache describe-cache-clusters
  > aws elasticache describe-cache-clusters --show-cache-node-info
  > aws elasticache delete-cache-cluster --cache-cluster-id dockerzon-production (if want to delete)

alt text

Setting up an Elastic Load Balancer

  > aws ec2 describe-subnets
  > aws elb create-load-balancer --load-balancer-name dockerzon-web --listeners "Protocol=HTTP, LoadBalancerPort=80, InstanceProtocol=HTTP, InstancePort=80" --subnets subnet-a03XXXX subnet-76XXXXX --security-groups sg-c78XXXX
  > aws elb describe-load-balancers
  > aws elb modify-load-balancer-attributes --load-balancer-name dockerzon-web --load-balancer-attributes "{\"ConnectionSettings\":{\"IdleTimeout\":5}}"
  > aws elb configure-health-check --load-balancer-name dockerzon-web --health-check "Target=HTTP:80/health_check, Timeout=5, Interval=30, UnhealthyThreshold=2, HealthyThreshold=10"
  > aws elb delete-load-balancer --load-balancer-name dockerzon-web (if want to delete)

alt text

Visualizing the Application's Architecture

alt text

Profiling the Ruby on Rails Application

  > cd dockerzon
  > add RAILS_ENV=production in .dockerzon.env top
  > docker-compose up -d
  > docker exec dockerzon_dockerzon_1 rake db:reset
  > curl https://127.0.0.1:8000
  > docker stats dockerzon_dockerzon_1 dockerzon_sidekiq_1
  > docker pull williamyeh/wrk
  > docker run --rm williamyeh/wrk -t10 -c50 -d10s https://<machine_ip>:8000
  > docker-compose stop
  > revise WEB_CONCURRENCY=2 in .dockerzon.env
  > docker stats dockerzon_dockerzon_1 dockerzon_sidekiq_1
  > docker run --rm williamyeh/wrk -t10 -c50 -d10s https://<machine_ip>:8000

alt text

Deploying Everything with Amazon ECS

Introduction

alt text alt text

Creating the Production Cluster

  > cd production
  > aws ecs create-cluster --cluster-name production
  > aws s3api create-bucket --bucket ecs-dockerzon --region ap-northeast-1 --create-bucket-configuration LocationConstraint=ap-northeast-1
  > aws s3 ls s3:https://ecs-dockerzon
  > aws s3 cp ecs.config s3:https://ecs-dockerzon/ecs.config (upload)
  > aws s3 ls s3:https://ecs-dockerzon

Creating the Private Registry Repositories

  > aws ecr get-login --no-include-email --region ap-northeast-1
  > aws ecr create-repository --repository-name dockerzon/dockerzon
  > aws ecr create-repository --repository-name dockerzon/nginx
  > aws ecr describe-repositories
  > docker tag dockerzon_dockerzon:latest 28XXXXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/dockerzon/dockerzon:latest
  > docker push 28XXXXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/dockerzon/dockerzon
  > docker tag dockerzon_nginx:latest 28XXXXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/dockerzon/nginx:latest
  > docker push 28XXXXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/dockerzon/nginx

Spinning up Multiple Container Instances

  > cd production
  > aws ec2 run-instances --image-id ami-21815747 --count 3 --instance-type t2.micro --iam-instance-profile Name=ecsInstanceRole --key-name aws-ice --security-group-ids sg-c78xxxxx --user-data file:https://copy-ecs-config-to-s3
  > aws ecs list-container-instances --cluster production (--key-name mean ssh key to access ec2)

Registering the Task Definitions

  > cd dockerzon
  > docker-compose up -d
  > docker exec dockerzon_dockerzon_1 rake secret
  > cd production
  > aws ecs register-task-definition --cli-input-json file:https://worker-task-definition.json
  > aws ecs register-task-definition --cli-input-json file:https://web-task-definition.json
  > aws ecs register-task-definition --cli-input-json file:https://db-reset-task-definition.json
  > aws ecs run-task --cluster production --task-definition db-reset --count 1
  > aws ecs deregister-task-definition --task-definition db-reset:1
  > aws ecs register-task-definition --cli-input-json file:https://db-migrate-task-definition.json
  > aws ecs run-task --cluster production --task-definition db-migrate --count 1
  > aws ecs list-task-definitions

Scheduling Services

  > cd production
  > aws ecs create-service --cli-input-json file:https://web-service.json
  > aws ecs describe-services --cluster production --services web
  > aws ecs create-service --cli-input-json file:https://worker-service.json
  > aws ecs describe-services --cluster production --services worker
  > curl <aws_elb_dns_url>

Pushing Application Changes without Downtime

  > cd dockerzon
  > docker-compose build
  > docker tag dockerzon_dockerzon:latest 28XXXXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/dockerzon/dockerzon:latest
  > aws ecr get-login --no-include-email --region ap-northeast-1
  > docker push 28XXXXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/dockerzon/dockerzon
  > curl <aws_elb_dns_url>
  > cd production
  > aws ecs register-task-definition --cli-input-json file:https://web-task-definition.json
  > aws ecs update-service --cluster production --service web --task-definition web --desired-count 2
  > aws ecs register-task-definition --cli-input-json file:https://worker-task-definition.json
  > aws ecs update-service --cluster production --service worker -task-definition worker --desired-count 1
  > curl <aws_elb_dns_url>

Automating Your Deployments with a Script

  > cd production
  > ./deploy.sh -h
  > ./deploy.sh -a

Exploring the AWS Console and Container Logs

  > ssh -i "aws-ice.pem" ec2-user@<ec2_ip or ec2_dns>
  > docker container ls
  > docker container ls -a
  > docker logs -f <container_id or name> (watch log)
  > docker logs --tail 20 <container_id or name> (get tail 20 log)

Domain Names and SSL

Creating Free SSL Certificates with Amazon ACM

  > aws acm request-certificate --domain-name dockerzon.XXXXX.cc
  > aws acm describe-certificate --certificate-arn arn:aws:acm:ap-northeast-1:28XXXXXX:certificate/fab62309-4979-4805-b0a5-XXXXXX
  > aws acm list-certificates
  > aws acm delete-certificate --certificate-arn arn:aws:acm:ap-northeast-1:28XXXXXX:certificate/fab62309-4979-4805-b0a5-XXXXXX (if want delete)

Updating the Security Group to Handle SSL

  > aws ec2 authorize-security-group-ingress --group-id sg-c78XXXX --protocol tcp --port 443 --cidr 0.0.0.0/0

Updating the ELB to Handle SSL

  > aws elb create-load-balancer-listeners --load-balancer-name dockerzon-web --listeners "Protocol=HTTPS, LoadBalancerPort=443, InstanceProtocol=HTTP, InstancePort=80, SSLCertificateId=arn:aws:acm:ap-northeast-1:28XXXXX:certificate/fab62309-4979-4805-b0a5-XXXXXX"
  > aws elb describe-load-balancers --load-balancer-names dockerzon-web

Updating nginx to Handle SSL

  > cd nginx
  > docker build -t dockerzon_nginx .
  > cd production
  > docker tag dockerzon_nginx:latest 28XXXXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/dockerzon/nginx
  > aws ecr get-login --no-include-email --region ap-northeast-1
  > docker push 28XXXXXXX.dkr.ecr.ap-northeast-1.amazonaws.com/dockerzon/nginx
  > aws ecs register-task-definition --cli-input-json file:https://web-task-definition.json
  > aws ecs update-service --cluster production --service web --task-definition web --desired-count 2
  > curl <https_aws_elb_dns_url>

About

Learn how to build and deploy a fault tolerant, scalable and load balanced AP on AWS ECS.

Topics

docker aws ec2 s3 ecs rds ecr ecs-cluster scaling-aws-ecs

Resources

Readme
Activity

Stars

6 stars

Watchers

3 watching

Forks

4 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages