Why CloudFormation is needed in a nutshell
- Infrastructure as Code service by AWS. A complex application can have so many resources dependent on each other. CloudFormation helps you to manage all of those resources in the form of template. You can create as many copies as you want of this template. E.g. multiple regions for backup purposes, create in one region and deploy on others. Not only managing the resources, you can manage the entire lifecycle of an application.
Structure of JSON
- AWS Template Format Version: the version of that template.
- Description: plain text explaining what the script does for that particular template.
- Metadata: containing the properties of the template, stack version etc.
- Parameters: any values you have to pass to the template will be included in the parameters.
- Mappings: include the dependencies between your resources. E.g. when you need to use a specific AMI from a specific region, you recall that AMI via Mapping.
- Conditions: when stack is being created, any updates or conditions given here will be evaluated. For example you can use a WaitCondition for following situations;
- To coordinate stack resource creating with configuration actions that are external to the stack creation
- To track the status of the configuration process
- Outputs: output message is given.
- Resources: any kind of AWS Resources that you may want to create needs to be included here. This is the most important part of the JSON script.
CloudFormation Key Facts
- CloudFormation plays very well with most of the pipeline tools and technologies.
- As your group of resources get deployed and you update those resources. Suppose one of the updates was a failure, CloudFormation keeps your state and it is able to rollback to the last known state.
- CloudFormation knows about stabilisation of the resources.
- CloudFormation now has integration with 421 different resources within AWS.
- Stack operations will generate event streams or you can even define SNS topic to integrate with other services such as emails, service management etc. to link your stack.
- CloudFormation takes logical names of the resources you define within the stack definition and then matches those with the actual physical IDs of the resources one they are created.
- Stack Updates: You can either change the whole template you created to update the stack or you can even change the parameters used in your template to update resources.
- CloudFormation double-checks to identify a new addition of a resource within the updated template and goes ahead to create the additional resource. If you replace the original template once again, CloudFormation now will delete the bucket.
- Updating the config of existing resources: you can either make changes directly or changing the parameters.
- Change set: if you want to change the instance type and even the keys, CF needs to create a new instance with the keys provided. It’s useful to track change control within the infrastructure. A separate member of the IT team can review all change sets for a given environment and get SNS emails as well. It will show the change type (modify etc.), logical and physical ID and the required action (replacement for example).
- Update Behaviors of Stack Resources
- Update with no interruptions: Updates without any disruption.
- Updates with some interruption: For example; if you change the instance type of an EC2 instance, CF shuts down that instance, resizes and reboots again. Hence, you would expect some delays.
- Replacement: CF creates the replacements first, changes references from other dependent resources to point to the replacement one, and then deletes the old resource. For example; engine property of a RDS instance. That can result in data loss as well.
- Template Reuse and Portability: You should be able to reuse the template unlimited number of times and at any region and at any account without any restriction.
- Pseudo parameters: you can dynamically refer to things such as account, region, partition etc. in order not to hard code specific resources
- Intrinsic Functions: For example; Fn::GetAzs will get the list of AZs within the region you specify. You can specify hardcoded regions, current region or even referencing the pseudo parameter of the region.
- If you don’t specify, for example a bucket name for S3 bucket resource, when creating the portable template, CF creates a physical ID of that resource intelligently, by appending the stack name with resource name and a random key.
- Nested Stacks and Cross Stack Reference: This is important for the lifecycle of your CF stacks. An example of this would be a VPC stack. You can reference the configuration of certain set of resources from another stack. You can define an output as the result of CF execution. For example, VPC and the each subnet. You can define the additional JSON block unique to the region and the account. Then you can get that block and refer it elsewhere by using the Fn::ImportValue in a different template and stack within a different region.
- Cross Stack Reference is useful to avoid hardcoding values in your templates. You apply cross-stack because stacks you deploy are active and you want to reference resources from one another.
- Nested Stacks: You use in an attempt to re-use blocks of CloudFormation templates. The stacks you would create will have a dependency chain. For example, you need to do VPC stack first to be able to create an AD stack and then finally you would implement the RDP stack to connect to the domain controller instances. For example; you define the VPC logical name, and provide the Template URL for the stack from S3. This creates a different stack without referencing and can be re-used over and over again. Compared to Cross Stack Reference, Nested Stacks create a new set of resources based on the templated used within the stack.
- Stack Roles: By default, CloudFormation uses the default permissions of the person interacting with the stack. You can also give an IAM role for the stack for any additions, updates etc. You are able to allow junior staff members to interact with the stack on your behalf with limited permissions. It would allow them to manage resources without using the account credentials.
- Stack Sets: It allows you to create an administrator account, target account and deploy stacks in different regions at different accounts. You have to define a trust between the AWS accounts. You need to create 2 different IAM roles. First one is AWSCloudFormationStackSetAdministrationRole and the other role in the target account will be AWSCloudFormationStackSetExecutionRole. You might have shared VPC architecture template, standard and you want to deploy in every region or if you want to deploy pre-made AWS templates. Some of the examples existing on AWS are;
- Enable AWS CloudTrail at every account
- Enable AWS Config
- Add config rule root-account-mfa-enabled
- Disaster Recovery with CloudFormation: It’s the cheapest form of DR but also the slowest. You can use CF as part of a DR strategy. Rapidly scalable backup site deployment can be achieved with CloudFormation. You can re-hydrate from a backup or data dump or even re-size the infrastructure.
- Custom Resources: You might want to interact with resources outside of AWS, especially when building a hybrid architecture. You define a custom resource within your template via either an SNS backed resource or a Lambda backed resource. CF sends a data block to the external entity to wait for a response to acknowledge the resource had been successfully created. CF will send out event/notification to Lambda or SNS and the external entity needs to act on it.
Provisioning at Scale with CloudFormation
- StackSets: Create and update stacks in multiple accounts and regions using a single operation. By this, you would be able to enable CloudTrail for all regions, use admin’s S3 buckets, set up AWS Config rules to properly tag resources and set up AWS KMS keys. This way, you can deploy immutable and repeatable stacks across regions and different accounts.
Protecting your Stack
- Lock your stack: you can enable termination protection radio button. Once your stack is created, you will see the protection enabled on the overview section. When you try to delete the stack, API call will be disabled/failed. This is ideal for critical stacks.
- Stack level policies: Example is to only prevent updates to all RSD databases instances. You can attach when creating or updating a stack. Deletion policy: there are type of resources that you don’t want to delete at all.
- Resource by resource
Monitoring Stacks and Resource Changes
- Detecting configuration drift: Tracking changes made outside of CloudFormation to stack Resources. Let’s say you created JSON/YAML templates, suppose you created web app enterprise app and database. Then you will have a team of engineers/teams trying to change the stack via different routes, out of band. Any changes made outside of AWS CloudFormation to one or more resources contained in the stack that modify the expected configuration values of resources would cause drift in the stack.
- Stack update might fail because of a drift.
- Audit and compliance: divergence from your approved architecture.
- Use the diff viewer in the console to pinpoint resource changes.
- Check particular stack or resource and you can combine with AWS Config to periodically check your stack configuration.
- Dynamic monitoring: Monitoring during stack changes.
- Recording changes
Record & Monitor Stack Changes With AWS Config
- Track current and historical stack configuration.
- Use case of audit & governance.
AWS Private Link
- Since Private Link is designed to keep all the network traffic within AWS network internally, you can use CloudFormation APIs inside of your VPC and route data between your VPC and CloudFormation entirely within the AWS network. There is no need for proxies, NATs or Internet Gateway.
Template Schema Plugin
- When you type AWS::EC2::, you will know what else needed in terms of syntax for each type of resource you are planning to add within your stack.
Pipeline in CloudFormation
- From source to build, test and promote.
- Let’s say you are a developer, you created your Template, then commit it to your source control tool. Source your code all the time. From there, you build and test via cfn CloudFormation. You can do some unit tests with Lambda as well.
- You can create an iterator within a serverless function (Lambda), transform which is a macro. E.g. I wanna spin up 3 S3 buckets. You can make it simple as you want it to be.
- Execute Python: you can arbitrarily run python codes within your CloudFormation templated. E.g. is generating tags for your resources.