Terraform Tutorial – Getting Started With Terraform on AWS

Terraform is a product by Hashicorp that uses Infrastructure as Code (IaC) to provision cloud infrastructure.

In this tutorial, we will learn step-by-step how to use Terraform, enabling you to manage cloud infrastructure with IaC.

1. The Benefits of Using Terraform
2. Terraform Features
3. Installation and Setup
4. Terraform Providers
5. Resources
6. Terraform CLI
7. Terraform Variables
8. State Management
9. Remote Backends

Traditionally, without IaC, the cloud infrastructure was managed manually. This was not the most efficient way and was prone to manual errors. Consistency was a challenge, especially when many servers and clusters were to be managed.

Configuration management tools existed, but the infrastructure’s support was very limited. Application code development and management of that code has evolved a lot with versioning tools, DevOps toolchains, development practices, and delivery methodologies. Terraform introduced the concept of Infrastructure as Code, which by default leverages these advantages to managing infrastructure. 

Let us take a look at a few of the benefits Terraform has to offer.

You can also take a look at 9 Popular Terraform Alternatives.

Note: New versions of Terraform will be placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that will expand on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6. OpenTofu retained all the features and functionalities that had made Terraform popular among developers while also introducing improvements and enhancements. OpenTofu works with your existing Terraform state file, so you won’t have any issues when you are migrating to it.

Some of the key features of Terraform are

With that in mind, let us get our hands dirty by installing and setting up our Terraform environment.

Terraform

Terraform comes in two forms: open source and Terraform Cloud. Terraform Cloud is an online hosted platform that provides a UI for automation of provisioning tasks and management, but the code still needs to be manually developed.

You can also check out Spacelift, a sophisticated and compliant infrastructure delivery platform that makes Terraform management easy.

For the sake of this tutorial, we would work with an open-source version that offers all of the same features for self-hosting. The Terraform binary can be downloaded from the Terraform website.

Choose the OS and package manager or binary download option as per the choice of your system. The screenshots in this tutorial are based on macOS. Apart from the setup, the steps described in this tutorial are the same across all the operating systems.

If you need more details, see a complete guide on downloading and installing Terraform on Windows, Linux, and macOS.

The installation steps generally consist of downloading the appropriate binary and setting up the path variable. Once installed successfully, test if the installation works by checking the version below. When writing this tutorial, the latest version of Terraform was v1.2.3.

The code for this tutorial is available at this github repository.

AWS CLI

Since we will be working with AWS to provision our infrastructure using Terraform, we also need to install AWS CLI. Follow the steps listed here, to install the latest version of AWS CLI for your OS. After a successful installation, check the version of AWS CLI as shown below. When writing this tutorial, the latest AWS CLI version was 2.7.9.

Terraform uses AWS CLI to make API calls to execute any provisioning task. Log in to AWS Web Console and create a user for Terraform. The user needs programmatic access to AWS, so check the corresponding checkbox only.

For now, provide the administrative role to this user. However, it is recommended to follow the principle of least privilege when working on projects.

Once the user is successfully created, take a note of the Access Key ID and Secret Access Key.

We need to configure these credentials in the AWS CLI we just installed. Open the terminal and run the aws configure command to configure default credentials. I have left the region as the default value since I am using the same. Select the region of your choice if required.

With this, we have successfully set up our environment to begin working with Terraform and AWS.

Note: Terraform is able to make API calls without AWC CLI if you hardcode credentials or use some other method to access AWS.

Terraform implements a modular approach in its application architecture. The Terraform binary we downloaded is the core module required to perform core Terraform functions. Any operation or a CLI command that does not involve invoking any cloud provider API are core functions of this binary.

To work with a cloud provider, AWS in our example, Terraform instantiates a corresponding module. It downloads the binary in the root directory of the project. We instruct Terraform to use a specific version of the AWS provider so that our Terraform project can provision AWS resources.

Now is the time to create our Terraform project. Create a directory in your system and open this path into an IDE of your choice. All the Terraform code goes into .tf files created in the root of this directory. Create the first file named provider.tf into this empty directory and write the provider block as below.

As mentioned before, Terraform uses HCL syntax. A declarative configuration language that helps us declare the cloud resources we want to provision using Terraform. In the provider.tf file, we have specified a terraform block, within which we have declared another block that represents required_providers. 

required_providers contains an attribute aws to which an object with a couple of properties is assigned. These properties define the source and desired version of the AWS provider. Provider-specific documentation is maintained in the registry. To check the latest version of the available provider and configure various types of resources with AWS, refer to the Terraform registry.

The above code instructs Terraform to initialize the AWS provider with version 4.19.0. Save the provider.tf file. To initialize our Terraform project, run terraform init command in the root directory and observe the output as below.

As we can see, Terraform has successfully initialized the desired version of the AWS provider. If we look at our root directory, here is what it contains.

Apart from provider.tf and README.md file, we have .terraform.lock.hcl file and a sub-directory named .terraform. The lock file here is used to manage checksums for the downloaded module binaries, which we will discuss in the next sections. .terraform sub-directory is the target where AWS provider plugin binary is downloaded. If you have followed the steps successfully till now, this is how your directory should look.

At this point, we have set up the Terraform provider, and now we would move on to create some resources in our AWS console using Terraform.

Create another file named main.tf in the same directory. The intention of creating this file is to declare the resources we want to create in AWS. It is completely okay to skip creating this file and just append provider.tf file with the below code. However, creating separate files helps manage the Terraform code in a better way. Add the below code to main.tf file.

Here we have declared a resource block of type “aws_instance”. This instructs Terraform that we want to create an EC2 instance resource in AWS with given attributes. aws in aws_instance helps Terraform identify the provider it would require to perform this provisioning task.

Note: To identify the correct resource type and associated attributes, refer Terraform Registry.

The second parameter is “my_vm”, which is an internal identifier. Its purpose is to refer to this particular EC2 instance elsewhere in the code. We can assign any name to this identifier.

Amongst the resource attributes, we have declared the ami – which defines the image we want to use to create this EC2 instance. The chosen AMI here is an Ubuntu image in Central Europe. We have also specified the instance_type to define the size of the EC2 instance to be created.

Additionally, we have assigned a tag “Name” with the value “My EC2 Instance”.

With this, we have successfully created the code to create an EC2 instance in AWS using Terraform.

We have created the Terraform code to provision our AWS resources, but we have not yet provisioned them. This is what we would do in this section. Let us take a look at some of the important Terraform CLI commands, that also help us understand the resource lifecycle managed by Terraform.

Format (fmt)

Assuming we have already written some Terraform code with correct syntax, it may still lack proper indentations. A properly indented code is very valuable as it improves readability. terraform fmt command helps us format the Terraform code for all .tf files in the given directory.

In your project directory, where you have created the main.tf and provider.tf files, run terraform fmt to achieve well-formatted Terraform code in microseconds.

It outputs a file name – main.tf. On closer observation, in the main.tf file, in the code we created before, the attributes – although aligned as per the positioning of “=” sign, there are unnecessary spaces between the name and the “=” sign. Now that we have formatted the code, these spaces are removed.

The output just represents a list of all those files which are reformatted by fmt command.

Initialize (init)

We have already worked with this command before. Usually, this command is run only at the beginning of a Terraform project – as far as we stick with the same provider. However, if we decide to add more providers, or change the version of existing ones and try to execute the Terraform code without initializing – it throws an error. For new developers, this is a common step that is missed.

In our provider.tf file, change the version attribute of aws from “4.19.0” to “4.18.0” and run terraform plan command in the console.

As it must be clear from the error message, Terraform identifies this change and asks us to initialize this directory again to update the appropriate provider module. Run terraform init -upgrade now.

Thus, running init command with the -upgrade flag has helped us match the version of the provider module to be used.

Plan (plan)

Plan command helps us validate the resource declaration we have done using Terraform. It takes into account all the Terraform configuration files and identifies any syntax errors, version miss-match errors (like above), etc.

Additionally, it validates the code with the state file (discussed further) to identify and highlight resources that are to be created, updated, or deleted if we choose to execute the current version of the code. It is a great way to have a snapshot of changes before we apply them to the target set of infrastructure.

In our code directory, run terraform plan and observe the output.

There is quite a lot of information provided in the output here. In summary, it highlights all the resources which will be created. Since this is the first time we are going to run the code – there are no changes or deletions highlighted. The plan summary at the end of the output is a good way to match the stats when we deal with a larger set of resources.

Read more about Terraform security groups.

Apply (apply)

Now that we have verified the actions that Terraform will perform if we choose to execute this code, let us go ahead and create these resources. terraform apply command helps us provision all the resources we have declared in the main.tf file. When you run terraform apply, it again highlights the output similar to the plan command, and asks for our final confirmation as below:

Type in ‘yes’ and hit enter.

Terraform starts to provision the EC2 instance with the attributes we have specified. For all those attributes which are not specified, it assumes the default value. As seen from the output, it took 31 seconds for Terraform to successfully provision our EC2 instance. Verify the same by logging into your AWS console.

Under the hood, Terraform uses the AWS provider plugin, which in turn makes programmatic API calls to AWS using the credentials we configured during setup.

Congratulations, if you have managed to reach this point!

Destroy (destroy)

Terraform manages the lifecycle of infrastructure. This also means, that if it is responsible for the creation of a particular resource, it is also responsible for its deletion or destruction. Terraform destroy command helps us delete all these resources completely.

To delete the EC2 instance created using apply, run terraform destroy to terminate the same instance as below.

Here, it again asks for our confirmation with an output that highlights the resources which will be deleted. If we choose to say anything other than ‘yes’, Terraform will abort this operation. Since the plan summary correctly indicates that one resource will be destroyed – which is as per our expectations – let us go ahead and confirm the deletion.

As we can see, Terraform has successfully deleted the EC2 instance. Verify the same from the AWS EC2 console.

Till now we have been able to create and destroy AWS EC2 instances using Terraform. Terraform commands we just discussed are the most important commands which every developer should be aware of. These commands are vital, in the way that they define the complete end-to-end lifecycle of the resources being managed via Terraform.

Variables plan an important role when dealing with Terraform projects. So far we have hardcoded all the values like AMI, Instance Type, and Tags, required to provision our EC2 instance. Imagine doing this for multiple resources – EC2 instances with various configurations, RDS instances, VPCs, etc. – this approach can quickly become cumbersome.

Variables introduce the much-needed flexibility and dynamics required to manage larger sets of infrastructure. Variables also play an important role when applying the DRY (don’t repeat yourself) principle when creating modules. We will discuss modules in the upcoming sections.

There are mainly three types of variables – local, input, and output. We will discuss this one by one.

Local variables

As the name suggests, these are locally declared variables. These variables provide a name for any attribute value that is to be used anywhere in the Terraform code. Local variables are especially useful if we were to refer to these values in multiple places in our Terraform code. We can perform various operations to calculate these values based on multiple other values as well.

For the sake of our example, we keep it simple. Create a file named variables.tf in the root directory and declare the local variables as below.

Here we have declared all the hardcoded values into their corresponding local variables. Local variables, once declared, can be referred to anywhere within the module. To refer to these variables in our main.tf file, use the syntax local.<variable_name> at appropriate places. After the changes are done to main.tf reflect usage of variables in main.tf file, it should look like below.

To verify if we have correctly declared and referred local variables in our code, run terraform plan and observe that it does not throw any error. At this moment, it is enough to rely on the planned output for verification.

Input Variables

At times, there are situations where the values are to be provided to for the execution of code during run time. Local variables only work locally, they lack the ability to accept and assign values dynamically during runtime. 

Input variables provide this function. This ability also helps us define input values for modules to accept values on the go from the parent module.

Let us redefine our variables.tf file with input variables instead of locals for the same attributes.

We have created three variables – all of the type string – and provided them with descriptions. Description of input variables helps in generating documentation if we decide to publish this as a module on Terraform registry. Do the corresponding changes to main.tf file to use input variables instead of locals.

When we run terraform plan command at this point, it validates the code successfully. However, try to run terraform apply and observe the output.

Here Terraform asks to provide the value for ‘Ubuntu AMI ID’, and it would ask the same for the other two variables consecutively. This is just to prove that Terraform allows us to provide these values dynamically during runtime.

If we want to avoid supplying values one by one in CLI, we can declare default values in the same variables.tf file as shown below.

Now if we run terraform apply, it will automatically assume the default values provided in this file and proceed towards provisioning of the EC2 resource.

A better way to manage these default values is to create another file named terraform.tfvars. Terraform automatically interprets this file as a group of key-value pairs and maps them with the declared variables in the variables.tf file.

Clear out all the default values from variables.tf file and create terraform.tfvars file as shown below. The syntax is quite straightforward – on every line, there is a key assigned with a value.

Let us try to run terraform plan command at this point. As we can see, Terraform indicates a successful plan output.

Note: The name terraform.tfvars is a prerequisite. If we choose any other name before .tfvars extension then the file needs to be explicitly supplied in the CLI command with the -var-file flag. Optionally, we can also use the .auto.tfvars extension so that Terraform can pick this file automatically to read variable values.

Output Variables

Till now we have been able to create EC2 instances using Terraform code and CLI commands. However, to get the details about that EC2 instance we log in to the AWS Console and see the required attributes.

Output Variables provide a way to retrieve the details we are interested in, in the same CLI terminal. Additionally, output variables help “return” values to the parent modules for further processing. 

We define output variables in the same variables.tf file. In general, you can define Output or Input variables in any .tf file and they will be interpreted by Terraform as far as they reside in the working directory.

Let’s say we are interested in knowing the public IP and Instance ID of the EC2 instance which will be provisioned. We declare the output variables as below. Append the below code in the variables.tf file.

Run terraform apply and observe the output.

The plan output now also indicates the values which will be known after apply. It chooses to display these values because we have declared their corresponding output variables. Add more output variables to know more about your instance after provisioning.

Run terraform apply to see the values being generated.

As we can see, Terraform has successfully provisioned the EC2 instance and we also have the instance ID and public IP information handy. We did not have to log in to the AWS console to find the same.

Note: As far as the naming convention of the files is concerned, apart from .tfvars, there is no constraint for naming these files like we have done until now. You could name them as per your wish – Terraform considers all the files with .tf extension in any given directory. The names provider, variables, and main are generic conventions used in the ecosystem.

State management in Terraform is one of the crucial things to know and learn about especially when working with teams. In this section, we dive deep into understanding how state files in Terraform work and the impact it creates on the resource lifecycle management workflow, which then offers a great introduction to the next section about remote backends.

Until now we have created multiple files and if we look at our root directory, the system/Terraform seems to create a few more files. Now is the right time to revisit them.

We have already discussed the .terraform directory, .terraform.lock.hcl file, main.tf, provider.tf, variables.tf, and terraform.tfvars files. By now we know the purpose of these files. The two new files created are –

1. terraform.tfstate 
2. terraform.tfstate.backup

As a quick recap of our journey till now, we have set up the Terraform environment and declared the configuration/code to create an EC2 instance in AWS. We also destroyed the resource which we created in this process. We have seen how Terraform performs these operations under the wraps and updates the status in the CLI terminal.

Notice how Terraform only works with the resources created using Terraform alone. AWS resources that are created using Terraform – only those resources are destroyed. If we had multiple EC2 instances existing before the one created by Terraform, those resources will not be affected.

Terraform manages only those resources that the code written by us creates. For this, Terraform maintains the mapping information regarding the resources defined in the configuration and their corresponding real-world cloud entities. This mapping is saved in the state files.

Terraform.tfstate file thus maintains this mapping which in turn helps Terraform to identify the current state of the resources which are already provisioned, and the resources yet to be provisioned, or to be deleted.

When configuring infrastructure as code using Terraform’s declarative language, we essentially define the desired target state of the infrastructure. Any modifications to this code – addition or removal of resources – define the newer target state.

With Spacelift, you can get insight into application state, resource usage, changes, and more.

When the modified configuration is executed, i.e. terraform plan/apply command is run – Terraform compares the modifications made with the state files, for the corresponding existence in the real world.

If additional resources are found in the configuration, Terraform plans to provision those resources in the next apply command execution. Whereas, if lesser resources are encountered, Terraform identifies the removed resources by comparing the configuration with the state file and plans to destroy them.

Thus, the State file – terraform.tfstate – is an extremely important piece of information maintained by Terraform itself. The .terraform.tfstate.backup file – as the name suggests, the backup file holds the previous execution’s backup.

Note: Terraform CLI offers various commands to query and modify state information. These commands are directly related to this state file. It is recommended to avoid any manual changes as that approach is risk-prone. Always use the state manipulation commands for operations related to the state file.

State lock file

However, there is one more file involved in all of this – .terraform.tfstate.lock.info. We do not have it in the previous screenshot. Its existence is temporary and it only exists when plan, apply, or destroy commands are being executed.

To observe this behavior, run terraform apply in the root directory of our project and do not enter anything when Terraform asks for the confirmation step below.

Hold on to this moment, and observe the file directory.

Here we can see .terraform.tfstate.lock.info file is created automatically. If you now enter ‘yes’, this file exists for the duration until Terraform successfully provisions these resources. Similar behavior is observed for the destroy command.

The purpose of this lock file is to avoid race conditions. Imagine a scenario where Terraform environment is created in a shared VM and multiple developers try to apply their changes simultaneously. For Terraform to decide and execute the applied changes in a particular sequence, it uses this lock file.

You can stay on top of everything you’re responsible for with Spacelift – helping you visualize your resources, regardless of the technology used.

If we take a look at the lock file, it contains information in the JSON format as below. IT contains information about current execution – ID, Time when the execution is triggered, Who triggered it, Path to the state file etc.

Any request that is prioritized to be executed, creates this temporary information set – stored in terraform.tfstate.lock.info file – and temporarily puts on hold any other request till the current execution is completed.

Such simultaneous requests for Terraform’s execution is a very rare and rather non-suggested scenario, and the Terraform seems to have a fallback in the form of such a locking mechanism.

However, even if it is so when working in the team setup, this falls short of avoiding corruption in the resource configuration. In the next section, we will see why and how remote backends help address this issue.

The state files we discussed in the previous section reside in the same directory where the rest of the Terraform code is being developed. This is said to use a ‘local backend’. It works well if only one developer controls/contributes to the infrastructure being developed.

Another issue with working in local mode is – version control of the code itself. To leverage the advantage offered by version controlling systems like Git, it becomes risky to make state files part of the same remote git repository.

In a team where multiple developers are responsible for developing the Terraform code if we decide to commit the state file to the remote Git repo – causes variations in the state being stored locally by each developer. Multiple developers contributing to this repository can commit their own versions of state files. 

When pulled by other developers, it causes inconsistency. This results in corruption of state files and Terraform may end up creating unnecessary orphan resources or unnecessarily destroying required resources. This can cause a serious problem when the team is large enough and the set of infrastructure being developed is huge.

Additionally, state files may also contain sensitive information like credentials and keys, which can be risky to be stored in remote repositories.

Remote backends solve these problems. Using a remote backend essentially means using a separate storage service to securely store state files independently. Of course, this does not mean using remote Git repositories.

When we introduce a remote backend in the Terraform development workflow, the development may follow the below workflow.

1. Multiple developers have copies of the Terraform code from remote Git repositories available locally. They make their changes and push these changes back to the remote Git repository.
2. They execute terraform plan command from their local environment. This causes 
   • Terraform to apply the lock over the state file in the remote backend.
   • Query for state information.
   • Perform validation and present the plan.
   • Release the state file lock.
3. They execute terraform apply command from their local environment. This causes
   • Terraform to apply the lock over the state file in the remote backend.
   • Query for state information.
   • Identify the differences.
4. Terraform applies the changes in the configuration to the cloud provider.
5. Upon successful execution, Terraform releases the lock over the state file
   • This works similarly for the destroy command

In our example till now, we have been working with the local backend. We would now configure the remote backend to address shortcomings identified when using local file storage for state files.

A range of compatible backends are available and can be found here. For our example, since we are dealing with AWS, we would use S3 as our remote backend. This is not a necessity though, we can use any backend of our choice.

To use Amazon S3 as a backend, we need one S3 bucket. Let us add the configuration for the S3 bucket in our main.tf file as below.

Run terraform apply to provision this bucket in AWS. To use this bucket as backend, add backend "s3" block to the terraform block present in provider.tf file. The final contents of the provider.tf file should look like below.

If you try to run terraform apply it should throw an error. This is because we need to reinitialize Terraform since we have changed the backend information. Now when you run terraform init, it asks for confirmation to move the pre-existing “local” backend to the newly configured “s3” backend. This is exactly what we want. Type in ‘yes’ to confirm the same and hit enter.

If you see the output as above, it means we have successfully configured the given S3 bucket as our remote backend. Log in to the AWS console and verify the same.

Caution: If you now try to destroy the resources using the terraform destroy command, it will also delete the newly created S3 bucket – which is currently serving as our remote backend.

Run terraform destroy, and observe the output without confirming the same.

This is one of those cases where we would want to remove the bucket from Terraform management. The other option would be to create an S3 bucket separately and use it to configure the remote backend in the terraform block.

Removing the S3 bucket from Terraform management implies that we need to update the state file by removing the corresponding S3 bucket information. Doing this manually is risky. Ideally, the state files should never be touched. Terraform provides various state file management commands. One such command is terraform state rm command which removes the given resource from the state file.

From the terminal, run terraform state rm aws_s3_bucket.state_bucket.

Also remove the corresponding S3 bucket configuration from the main.tf file. Now if we try to run terraform destroy, it only destroys the EC2 instance. This way, the state file still remains intact and serves the purpose of our backend.

As we can see from the output, Terraform has only destroyed one resource, i.e. the EC2 instance.

If you want to learn more tips on managing S3 buckets with Terraform, check out our How to manage an AWS S3 Bucket in Terraform tutorial.

We have covered a lot of ground in this tutorial. We started from scratch, 

1. Installed Terraform and AWS CLI
2. Setup the AWS provider
3. Created and destroyed EC2 instance using Terraform
4. Worked with various types of variables
5. Worked with State files and understood their importance
6. Migrated our local backend to remote backend to enable team development

I hope this has been helpful for you. There are still some advanced concepts to be covered which we will cover in the next post.

If you need more help with Terraform, I encourage you to check the following blog posts: How to Automate Terraform Deployments, and 12 Terraform Best Practices.