The infrastructure-as-code (IaC) landscape offers so many tools, it can be hard to select the appropriate one. As well as multi-vendor, cloud-agnostic tools such as OpenTofu, Pulumi, and Terraform, we also have vendor-specific ones, like AWS CloudFormation.
In this article, we give you a solid overview of AWS CloudFormation, how it works, and the benefits it offers. We also discuss AWS CloudFormation templates, how to create and deploy them, getting started with AWS CloudFormation, and the best practices you should follow.
We will cover:
- What is AWS CloudFormation?
- What is an AWS CloudFormation service?
- What are AWS CloudFormation templates?
- What are CloudFormation sections?
- What is a CloudFormation change set?
- Benefits of AWS CloudFormation
- Getting Started with AWS CloudFormation – Tutorial
- What is an AWS CloudFormation hook?
- When should I use AWS CloudFormation?
- AWS CloudFormation best practices
- AWS CloudFormation alternatives
- Can I use CloudFormation with Spacelift?
AWS CloudFormation is an IaC tool developed by AWS to model, provision, and manage resources on the AWS cloud. CloudFormation can manage both AWS resources and third-party ones. As an IaC tool created solely for AWS and maintained by AWS, it integrates deeply with the AWS ecosystem, which means all changes, features, and new services are added immediately after they are released on AWS.
CloudFormation also has a very specific extension called SAM (Serverless Application Model), which can be treated as a framework to manage and deploy serverless applications to AWS with corresponding resources like S3 buckets or DynamoDB tables.
As an extension to CloudFormation, it is interpreted by the AWS CloudFormation service, even though it has a slightly different syntax. CloudFormation is also an AWS service that allows users to manage all stacks created with CloudFormation.
How much does AWS CloudFormation cost?
AWS CloudFormation is a free service. AWS does not charge for any usage of CloudFormation. Users only pay for services they used when they were creating the resources.
What is the difference between AWS Step Functions and AWS CloudFormation?
AWS Step Functions and AWS CloudFormation are two completely different services.
- AWS Step Functions is a service where users can orchestrate serverless workloads. The chain of actions and conditions is created and then managed by this service.
- AWS CloudFormation is an IaC tool. With it, users can create and manage infrastructure, including Step Functions.
Let’s explore key concepts of the AWS CloudFormation service, before moving on to AWS CloudFormation templates.
The CloudFormation service is a tool used to manage all operational aspects of CloudFormation and the infrastructure created by it.
The image below shows the service with some stacks already deployed:
CloudFormation Stacks
AWS CloudFormation Stacks are the templates deployed to AWS, using CLI, SDK, GUI, etc. They are a collection of resources created by the template, which is also part of the stack.
Through stacks, users can control and manage updates, executions, and termination of resources. It is important for users to be able to manage the behavior of a stack when it is configured. (For example, if an error occurs during deployment, what is the revert strategy?)
CloudFormation nested stacks
Cloudformation nested stacks are stacks created as part of the other stacks. Users can create multiple templates and reuse them in other ones. For example, a template used for creating an EC2 instance with its key and security group can be reused multiple times in the main template to create many EC2 instances.
CloudFormation StackSets
Cloudformation StackSets is an extension of Stacks functionality. It allows you to use a single template to deploy stacks in different AWS Regions and even in different AWS accounts.
CloudFormation Exports
It is quite common for one stack to depend on another one. CloudFormation Exports is the functionality that enables outputs to be passed from one stack to another. For example, when an EC2 instance is created, it should be run into a previously created Virtual Private Cloud (VPC).
CloudFormation Designer
AWS CloudFormation Designer is a tool that allows users to create, modify, or simply view the infrastructure created with CloudFormation. It can design the infrastructure and export it as CloudFormation code.
CloudFormation Registry
AWS CloudFormation Registry allows users to manage modules in the AWS space. Similar to Terraform, it allows you to create separate modules (for example to create an EC2 instance), store them in AWS CloudFormation Registry, and reuse them multiple times.
A CloudFormation template is a codified, declarative description of the resources and connections between resources that should be created in the AWS ecosystem. Templates can be created using JSON or YAML. They contain a few sections, which I will describe later in this post.
CloudFormation template sections
CloudFormation template sections are part of the template itself and have different roles. I will describe them in more detail below. One of the sections is Resources, where all resources (like VPCs, EC2, etc) are defined.
CloudFormation functions
CloudFormation functions allow you to automate and manage CloudFormation templates. The main functionality is the ability to use specific data while the value of this data is not available before the runtime. For example, if you need to collect a specific attribute from a freshly created resource, you should use the Fn::GetAtt: (or !GetAtt in short) function.
Read more: AWS CloudFormation Templates Tutorial
Sections in CloudFormation are the part of the code where a specific function is described. CloudFormation has nine sections:
- AWSTemplateFormatVersion
- Description
- Metadata
These first three sections are informational only. The most interesting is Metadata, as this section allows you to organize parameters in logical order. This helps later when CloudFormation is executed through GUI.
- Parameters allow users to provide data (similar to variables in Terraform), depending on the situation.
- Mappings allow you to provide a map of data values. For example, when the template is aimed at “Region agnostic” and covers multiple AWS Regions, you can provide the map of values specific for this region. One of the most common examples is AMI ID for EC2 instances. Another is the VPC ID per Region.
- Conditions. Unlike Terraform, CloudFormation expects to have clear, separately defined conditions, which will be used in the template.
- Outputs define the information that will be available from the stack. This works almost identically to Terraform outputs, but CloudFormation has additional functionality here: Users can export the output information and make it available as an input to another stack.
- Resources is the only obligatory section in CloudFormation. All resources are declared here.
- Transform is a very specific section where some macros can be provided, interpreted by CloudFormation, and “transformed” into a pure CloudFormation template. The best example is AWS SAM, an extension of CloudFormation that is interpreted by AWS during the execution through the Transform section.
Cloudformation change sets are the review/deployment concept that fits perfectly with CI/CD solutions. A change set contains two elements — create and deploy. Create provides visibility into changes in infrastructure caused by changes in template code. After review, these changes are deployed by change sets’ deploy functionality.
When a Cloudformation change set is generated, it contains all changes applied to the template. We can think of it as a delta between the deployed template and the updated one. This gives us the chance to examine the changes and decide if we want to proceed or not. Furthermore, we can track these changes in AWS in a more detailed way.
Another benefit of using change sets is the way it fits into CI/CD. In the CI part of the pipeline, we create an artifact — we execute aws cloudformation create-change-set command. This change set can be examined and analyzed by the process. And in the CD part, we deploy the artifact, using aws cloudformation execute-change-set command.
Why should I use change sets?
You may be wondering why you would use an approach where you have to use two commands instead of one. Let me underline the benefits:
- Fits CI/CD standard process
- Allows more granular control over implementation
- Enables you to perform quality checks
- Self-documentation. Change sets allow you to understand what changes were deployed by limiting the information to the change itself. It also can be used as the work done in tasks (in project planning tools).
AWS CloudFormation helps you to streamline your cloud infrastructure management. As well as offering a regulated, predictable way to build, edit, and delete your AWS resources, it offers several other benefits:
1. Price
AWS CloudFormation is a free service. You only pay to use the services required to create the resources.
2. Integration
The biggest benefit of CloudFormation is the fact that it is a native tool for AWS. It is deeply integrated with the AWS ecosystem, so everyone who uses AWS will be familiar with managing the service.
Another important aspect of CloudFormation’s inclusion in the AWS ecosystem is that organizations can create a self-healing approach, using AWS monitoring and compliance tools together with automatic triggers for CloudFormation.
3. CLI integration
CloudFormation is well integrated with the AWS CLI, which makes the operational processes easy.
4. Stability
This is a crucial long-term benefit. CloudFormation doesn’t provide breaking changes. Templates written many years ago are still executable.
5. Auditability
As part of the AWS ecosystem, CloudFormation can be configured under the control of AWS monitoring and governance tools, like CloudWatch, Amazon EventBridge, CloudTrail, and so on. This delivers a well-covered strategy of managing, controlling, and auditing all aspects of configuration changes. Additionally, specific actions can be triggered in case of defined situations, like detected drift or the creation of specific resource types.
6. Rich rollback functionality
CloudFormation provides multiple behaviors in case of failure. The default and probably the safest way is to revert all changes made in the current stack execution if failure occurs.
7. Use directly from the S3 bucket
With this functionality, you can have your template in VCS, of course, but you can add another layer of governance — a process that controls the quality of the template and, if all checkpoints are passed, shares the template in the S3 bucket and makes it available to teams.
Other general benefits of IaC that CloudFormation delivers include:
- Improved security. Putting operations on infrastructure through a well-controlled process with multiple quality check stages reinforces security.
- Repeatability and scaling. As IaC is a code stored in VCS, infrastructure can be scaled quickly both horizontally and vertically when needed. Repeatability helps the process because the action is to apply the code, not go through the whole process of creating infrastructure.
- Consistency. Code is committed and stored in VCS. This means that every execution will be done in the same way, with the same guardrails applied.
Another benefit of CloudFormation is that, if you have configured the work environment, you literally don’t need to do anything to start work with CloudFormation. You have your code editor installed, you have your AWS credentials configured, and you are familiar with AWS CLI. You are set to go.
How to execute CloudFormation?
AWS offers a few options to run CloudFormation:
- GUI. The standard way is to use an AWS GUI to deploy new stacks and manage existing ones
- AWS CLI. With AWS CLI installed, users can perform exactly the same operations as with the GUI.
- Different SDKs (like CDK or Boto3). This option requires knowledge of programming languages and is a different way of working with infrastructure.
How to create the AWS CloudFormation template?
The process is similar to other IaC tools. You can use predefined modules, or you can write your own template. The advantage of CloudFormation is its extensive library of existing, predefined templates.
Use predefined templates to create a CloudFormation template
We can use predefined templates in two ways:
- Download the template and provide all changes required.
- Templates can be used directly from AWS S3.
With predefined templates, you don’t need deep knowledge of AWS itself to quickly deploy simple applications — like WordPress in the example above.
Use the new template option to create a stack
A more popular option is to build and create the stack from the template prepared in your organization. This means the template is crafted to your needs and doesn’t contain unnecessary resources.
The template can be prepared with an AWS tool called Designer:
Designer allows users to visually create the resources and connections between them and then export them to a JSON or YAML template. Of course, Designer can also visualize existing templates.
However, the most common scenario is to create the CloudFormation templates using an integrated development environment (IDE) like Visual Studio Code, or even Vi. Below is a simple example of the CloudFormation code:
AWSTemplateFormatVersion: "2010-09-09"
Description: Setup CodeBuild and CodeDeploy pipeline
Parameters:
notificationEmail:
Type: String
Default: 'fake@email.com'
Description: Default email for pipeline notifications
Resources:
artifactsBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Delete
notificationTopic:
Type: AWS::SNS::Topic
Properties:
Subscription:
- Endpoint:
!Ref notificationEmail
Protocol: emailAfter the template file is prepared, it can be deployed to AWS.
How to deploy the CloudFormation template?
The most popular ways to deploy CloudFormation are through a GUI or CLI. We will see both in action soon, but there are some important things you need to know first about how the template is fetched:
Deploy the CloudFormation template directly from a local machine
The first option is to deploy directly from your machine (or directly from a CI/CD tool). We should, of course, keep our templates in VCS to ensure version control, but that step is independent of this process.
The file is fetched directly from the local directory and deployed on AWS. Naturally, AWS does some unseen work (for example, stores this template in a “hidden” S3 bucket), but from a user perspective, it looks like a direct deployment from disk to CloudFormation service.
The image below shows the simplified process with the AWS CLI (GUI works in the same way if we look at the big picture).
Deploy CloudFormation template through S3 bucket
In this option, we enforce better governance for our templates. The first step is to store the template in the AWS S3 bucket. Then, we deploy it using AWS CLI, but from this S3 bucket — not from a local directory.
This approach creates many new possibilities.
First, we create a CI/CD process to ensure the quality of the template. Only the templates that pass the quality checks will be copied to S3, ensuring that what is in S3 is free of issues and can be easily and confidently deployed to AWS. (In this case, S3 becomes an artifact storage.)
We can also build additional workflows using S3’s many integrations and triggers — for example, execute a Lambda function to generate some reports.
I strongly recommend the second approach because it is more mature and creates natural gates between the stages in DevOps SDLC.
Deploy CloudFormation template with GUI
Deployment using the AWS CloudFormation service GUI is very straightforward. This example is based on a WordPress template.
First, click “Create stack”:
You have the option to create a CloudFormation stack from existing resources, which is very handy if you decide to try the IaC approach and you already have working applications.
In my case, I selected the WordPress template to be created.
In the next step, I have to provide some parameters to configure my installation:
After accepting all default settings on the next screens, the stack is deployed:
We can observe the progress:
The CloudFormation stack is created almost immediately.
Deploy CloudFormation template with AWS CLI
I have AWS CLI installed and configured, so I will run the same example I used for the GUI.
It is important to remember that this template is parametrized, so we must provide values for the parameters. It is possible to do it in CLI command, but it is preferable to use the parameters file.
The file looks like this:
[
{
"ParameterKey": "KeyName",
"ParameterValue": "homeEc2FrankfurtDevOpsTraining"
},
{
"ParameterKey": "InstanceType",
"ParameterValue": "t2.small"
},
{
"ParameterKey": "SSHLocation",
"ParameterValue": "0.0.0.0/0"
},
{
"ParameterKey": "DBName",
"ParameterValue": "wordpress"
},
{
"ParameterKey": "DBUser",
"ParameterValue": "wordpress"
},
{
"ParameterKey": "DBPassword",
"ParameterValue": "wordpresspassword"
},
{
"ParameterKey": "DBRootPassword",
"ParameterValue": "adminpassword"
}
]Note: These settings are for example purposes only; you shouldn’t provide passwords this way and never leave ssh open to the whole world.
I am now ready to run the deploy command:
aws cloudformation create-stack --stack-name clistackforwordpress --template-url https://cloudformation-templates-eu-central-1.s3.eu-central-1.amazonaws.com/WordPress_Single_Instance.template --parameters file://parameters.jsonCLI returns the confirmation of action:
{
"StackId": "arn:aws:cloudformation:eu-central-1:1234567890:stack/clistackforwordpress/19892bf0-5eac-11ee-a50c-0aa515265511"
}To check the state of the stack I use describe-stacks:
aws cloudformation describe-stacks --stack-name clistackforwordpressAnd all information about the stack will be printed:
{
"Stacks": [
{
"StackId": "arn:aws:cloudformation:eu-central-1:1234567890:stack/clistackforwordpress/19892bf0-5eac-11ee-a50c-0aa515265511",
"StackName": "clistackforwordpress",
"Description": "AWS CloudFormation Sample Template WordPress_Single_Instance: WordPress is web software you can use to create a beautiful website or blog. This template installs WordPress with a local MySQL database for storage. It demonstrates using the AWS CloudFormation bootstrap scripts to deploy WordPress. **WARNING** This template creates an Amazon EC2 instance. You will be billed for the AWS resources used if you create a stack from this template.",
"Parameters": [
{
"ParameterKey": "KeyName",
"ParameterValue": "homeEc2FrankfurtDevOpsTraining"
},
{
"ParameterKey": "SSHLocation",
"ParameterValue": "0.0.0.0/0"
},
{
"ParameterKey": "DBPassword",
"ParameterValue": "****"
},
{
"ParameterKey": "DBName",
"ParameterValue": "wordpress"
},
{
"ParameterKey": "DBUser",
"ParameterValue": "****"
},
{
"ParameterKey": "DBRootPassword",
"ParameterValue": "****"
},
{
"ParameterKey": "InstanceType",
"ParameterValue": "t2.small"
}
],
"CreationTime": "2023-09-29T09:39:35.238000+00:00",
"RollbackConfiguration": {},
"StackStatus": "CREATE_COMPLETE",
"DisableRollback": false,
"NotificationARNs": [],
"Outputs": [
{
"OutputKey": "WebsiteURL",
"OutputValue": "http://ec2-3-121-234-56.eu-central-1.compute.amazonaws.com/wordpress",
"Description": "WordPress Website"
}
],
"Tags": [],
"EnableTerminationProtection": false,
"DriftInformation": {
"StackDriftStatus": "NOT_CHECKED"
}
}
]
}
{
"Stacks": [
{
"StackId": "arn:aws:cloudformation:eu-central-1:1234567890:stack/clistackforwordpress/19892bf0-5eac-11ee-a50c-0aa515265511",
"StackName": "clistackforwordpress",
"Description": "AWS CloudFormation Sample Template WordPress_Single_Instance: WordPress is web software you can use to create a beautiful website or blog. This template installs WordPress with a local MySQL database for storage. It demonstrates using the AWS CloudFormation bootstrap scripts to deploy WordPress. **WARNING** This template creates an Amazon EC2 instance. You will be billed for the AWS resources used if you create a stack from this template.",
"Parameters": [
{
"ParameterKey": "KeyName",
"ParameterValue": "homeEc2FrankfurtDevOpsTraining"
},
{
"ParameterKey": "SSHLocation",
"ParameterValue": "0.0.0.0/0"
},
{
"ParameterKey": "DBPassword",
"ParameterValue": "****"
},
{
"ParameterKey": "DBName",
"ParameterValue": "wordpress"
},
{
"ParameterKey": "DBUser",
"ParameterValue": "****"
},
{
"ParameterKey": "DBRootPassword",
"ParameterValue": "****"
},
{
"ParameterKey": "InstanceType",
"ParameterValue": "t2.small"
}
],
"CreationTime": "2023-09-29T09:39:35.238000+00:00",
"RollbackConfiguration": {},
"StackStatus": "CREATE_COMPLETE",
"DisableRollback": false,
"NotificationARNs": [],
"Outputs": [
{
"OutputKey": "WebsiteURL",
"OutputValue": "http://ec2-3-121-234-56.eu-central-1.compute.amazonaws.com/wordpress",
"Description": "WordPress Website"
}
],
"Tags": [],
"EnableTerminationProtection": false,
"DriftInformation": {
"StackDriftStatus": "NOT_CHECKED"
}
}
]
}How to update the CloudFormation stack
In the case of a stack update, CloudFormation offers two options.
Update the CloudFormation stack with GUI
First, we can update the stack directly by selecting the Update option from the GUI and providing all the changes (usually, it will be an altered version of the same stack template).
Update the CloudFormation stack with change sets
The second option uses CloudFormation change sets. This is preferable, as it allows you to control the process more closely.
AWS CloudFormation hooks are a relatively new functionality in CloudFormation that allows users to inspect the resources before actually implementing changes. To ensure all required resources are created, users can define a hook that will check the status of these resources.
For example, if I run a Lambda function that operates on a trigger from S3 bucket, I can use a CloudFormation hook before I create (or update) this function to check if my S3 bucket is created and is configured properly.
This is a multilayered question. If you have decided to use an IaC tool, you have to answer a few questions first:
- Do I need a cloud-agnostic tool as I use a multicloud approach? If the answer is yes, you should look at tools like OpenTofu, Terraform, or Pulumi, as CloudFormation is native to AWS and works with AWS only.
- Do I want to integrate deeply with the application’s code? If so, you should look at AWS CDK, which is CloudFormation under the hood but works as an SDK in many programming languages.
- Should I use as many tools from one vendor as possible? Although it might sound risky (as it might lead to vendor lock-in), this approach is very popular with organizations. If AWS is the cloud of choice, CloudFormation is a logical option.
CloudFormation best practices direct users on how to write, plan, and operate templates. AWS offers a full list of proposed best practices.
Let’s look at a few of them.
1. Reuse templates
Reusability is a core functionality of every IaC tool. We codify the resources for many reasons; one of them is the potential to take a piece and use it in another stack.
2. Manage and control IAM resources
Security and access management are very important, and wrong configuration may lead to breaches and incidents related to misconfigurations. Another reason for insisting on stringent IAM management is that only authorized people (preferably only automated tools) should have access to deploy the stacks.
Read more about IAM security best practices.
3. Do not hardcode sensitive information
As with any other automation tool, you should never hardcode sensitive information like passwords. Templates are stored in VCS, where many people may have access. Many security breaches have happened because hardcoded credentials were stored in public repositories.
4. Use CloudFormation change sets to update the stacks
This is worth repeating. Change sets allow you to control the update in a more manageable way and are invaluable for automated pipelines.
5. Enable AWS CloudTrail
AWS CloudTrail monitors all API calls to AWS. To apply some governance and compliance rules, CloudTrail should be enabled. When required, it will help monitor and analyze all actions applied by CloudFormation and also act on them automatically.
The DevOps and automation world offers multiple alternatives to most tools. CloudFormation is no exception.
1.CDK
CDK is an SDK approach where users can include creation and management of the resources in the application’s code. When executed, the CloudFormation will interpret this code.
2. SDK
The most popular SDK library is boto3 in Python. Similar to CDK, it allows you to add infrastructure parts into application code.
3. Pulumi
Pulumi is another SDK, deeply integrated with programming languages.
4. OpenTofu and Terraform
This is the most competitive approach to native solutions. Terraform is well established on the market, and OpenTofu is a new, open-source tool. Both facilitate managing multicloud IaC in one place.
To learn more, see our Terraform vs. AWS CloudFormation comparison.
5. Infrastructure from Code
In this emerging approach, code is decorated with expected behavior, and infrastructure is created based on these expectations during the runtime. This approach very often uses existing SDKs. One tool based on infrastructure from code tools is Klotho.
Spacelift can be configured with AWS CloudFormation as a backend. See the image below:
When you select the CloudFormation backend, you need to provide a little information — the AWS Region where the stack should be created, the stack name, the template file name, and the S3 bucket where the template will be uploaded and then executed.
All runs of the CloudFormation stack are completed with change sets.
Let’s take a look at the successful execution log in Spacelift:
You can see the Unconfirmed status on this image. I used this manual step here to show the moment where some review of the change set can be made.
AWS CloudFormation is a powerful IaC tool, native to AWS, and developed by AWS teams. It enables the DevOps approach to infrastructure, which is very deeply integrated with the AWS ecosystem.
CloudFormation templates can be created using Yaml or JSON and are managed by an AWS service called CloudFormation. This service allows users to fully control the IaC approach.
CloudFormation is not the only IaC tool in the DevOps landscape. The closest member of this family is CDK — the SDK based on CloudFormation. The main competitor is Terraform.
As a solution deeply integrated with AWS, CloudFormation contains some instruments that allow it to interact with AWS on a more detailed level than third-party tools.
Spacelift can also manage AWS CloudFormation, enforcing best practices, including change sets. It works as a powerful tool for organizations that use multiple tools to manage their infrastructure, for example, CloudFormation, Terraform, and Ansible.
If you are interested in reading more about AWS, you will find many articles on our blog.
Sounds interesting? Would you like to improve IaC management in your organization? Book a demo with our engineering team to discuss your options in more detail.
The Most Flexible CI/CD Automation Tool
Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.
