March 8, 2021

IaC or: How I Learned to Stop Worrying and Love AWS (sic)

Lets say you building something in AWS. A typical architecture - perhaps a 3-tier web-app: Load-balancer, app, and database. Maybe you want to use a container platform to run the app, so you will need ECS or EKS. Besides that, you will need VPCs, and subnets, and CloudWatch...‌‌This is how the build typically goes...lets go!

You used the console to create your container stack

  1. You created a VPC
  2. and an ALB, EFS
  3. Then the pipeline: CodeCommit, Build, Deploy
  4. then a cluster in ECS
  5. then had to figure out IAM policies

...and now you worried. Will you remember what you did, in what order? Will you be able to replicate it? Did you just adopt a new pet?

So you got tired of ClickOps, and tried the CLI

aws ec2 create create-subnet --vpc-id vpc-2f09a348 \
--cidr-block 10.0.1.0/24 \
--availability-zone eu-west-2a \
--region eu-west-2
aws elbv2 create-load-balancer --name ecs-ghost \
--subnets subnet-65abf80c subnet-72f65b3e \
--security-group sg-04a1a9b583455f819 \
--region eu-west-2

aws ecs create-cluster --cluster-name ghost --region eu-west-2

aws codepipeline create-pipeline --cli-input-json \
file://codepipeline-2-deploy-ecs.json —region eu-west-2

...and you still worried. There were many parameters that needed to be customised. Its gonna take time to rebuild this...still looks like a pet? The Ops team love it...but the Devs dont even know where to begin.

So you got smarter and tried the SDK...

import boto3

ec2 = boto3.resource('ec2')
# create VPC
vpc = ec2.create_vpc(CidrBlock='172.16.0.0/16')

import boto3
# Create ECS client
try:
  ecs_client = boto3.client('ecs')
  
  response = ecs_client.create_cluster(
    clusterName='CLUSTER_NAME'
  )
  print(response)

except BaseException as exe:
    print(exe)


So your developers are happy, because it looks like code...but they dont know what this VPC thing is, and how to link it to the cluster....so you still worried

Then you heard about CloudFormation!

image.png


After you got over the YAML shock, it looked awesome. You could model all your infrastructure, put them in stacks, and modify the stacks. So you started writing some CFN templates, and have lots of cattle!

To make it easier to write CFN for new resources, you can use ConsoleRecorder
But what about the existing resources you’ve built by hand? Use  Former2

But....your devs hated this YAML stuff. They need to know about VPCs, and Internet Gateways....and they wanted was a cluster to run their code.

Then you discovered CDK, and learned to stop worrying

This single CDK construct will build a VPC, ALB, ECS Cluster, IAM roles, pulls the container, and run it:

lbecs = aws_ecs_patterns.ApplicationLoadBalancedFargateService(self, 'ECS_Fargate', 
            memory_limit_mib = 1024,
            cpu = 512,
            desired_count = 1,
            task_image_options = ghost_task_image,
        )


Your devs love it, because it looks like code. They dont need to know about VPCs, and all that stuff. They just get what they want.  
How does it do that...this is how?

AWS CloudFormation was used to reliably and consistently provision the resources they needed, but the team discovered an unmet need. Although AWS CloudFormation was the right tool for provisioning resources, the team felt that using YAML/JSON was not the right approach for describing their system. AWS CloudFormation templates are basically a flat list of resources and their configuration. They don’t include tools for expressing abstract ideas such as “the injection pipeline” or the “storage layer” or a “dynamodb scanner.”

So why CDK?

  • Looks like code
  • Lives with your code, so its treated like code: git, IDE, linting, unit-tests, etc
  • Higher-level Constructs means it creates full services with sensible defaults, correct IAM privileges, and best practices
  • CDK == Well Architected (or close enough)
  • CDK Patterns takes it even further, building full use cases

Alternate Ending: AWS Copilot

https://aws.amazon.com/blogs/containers/introducing-aws-copilot/