[November 20 Webinar] Detecting & Correcting Infrastructure Drift

➡️ Register Now

Terraform

Terraform AWS Provider: How to Use & Examples

How to Use the Terraform AWS Provider

In this article, we will give an overview of what AWS provider is in Terraform and show everything from basic concepts to advanced configuration and authentication methods for managing AWS resources using Terraform.

What we will cover:

  1. What is a Terraform provider?
  2. What is Terraform AWS provider?
  3. How to set up the Terraform AWS provider?
  4. Terraform AWS provider configuration options
  5. Authenticating to AWS with Terraform
  6. Assuming IAM roles with Terraform AWS provider
  7. How to upgrade Terraform AWS provider to the newest version?
  8. Terraform AWS provider – Common issues and troubleshooting 

What is a Terraform provider?

A provider in Terraform is a plugin that enables communication with external APIs and services.  It lets Terraform interact with various cloud platforms, such as AWS, infrastructure providers, and other third-party services. Providers are specified within the Terraform configuration code, allowing users to define the specific services and resources they need to manage and provision.

For more details on Terraform providers (including the Spacelift Terraform provider), check out our previous article: Terraform Providers Overview.

What is Terraform AWS provider?

The Terraform AWS provider is a plugin that enables seamless integration between Terraform and the Amazon Web Services (AWS) cloud platform. It allows users to define, provision, and manage AWS resources using Terraform’s declarative configuration language. Before using this provider, you must configure it with the appropriate credentials to authenticate and authorize access to your AWS account.

Key features of the Terraform AWS provider

As for the majority of Terraform providers, the Terraform AWS provider comes with the following key features:

  • Resource management – create/modify/delete AWS resources based on your Terraform configuration, such as EC2 instances, VPCs, load balancers, S3 buckets, and others
  • Data sources – get information about existing AWS resources
  • State management with S3 and Dynamo DB – you can use AWS for remote state management using S3 to host the state and Dynamo DB for locking (Read more: How to Manage DynamoDB Tables With Terraform)
  • Great modules community – Because AWS is the leader cloud provider, it also has the biggest modules community

How to set up the Terraform AWS provider?

Let’s see the steps required to have the AWS provider up and running. The process for using the Terraform AWS provider is the following:

  1. Provider configuration
  2. AWS resource configuration
  3. Initializing Terraform to install the provider
  4. Running Terraform apply to create the resources

1. Configure the provider

You can configure your AWS provider in many ways listed here

Because Terraform can read information from your environment if you have set up your AWS credentials, you could configure the provider just by specifying the region:

provider "aws" {
 region = "eu-west-1"
}

If we don’t specify a Terraform block with version constraint for the provider, Terraform will automatically download the latest available version of the AWS provider.

At the time of writing, the latest version of the AWS provider is 5.51.1. Let’s add a Terraform block that makes us use the 5.50.0 version:

terraform {
 required_providers {
   aws = {
     source  = "hashicorp/aws"
     version = "5.50.0"
   }
 }
}

2. Define the AWS resources

Based on the provider above, we can configure AWS resources. For this example, we will just configure an AWS VPC:

resource "aws_vpc" "this" {
 cidr_block = "10.0.0.0/16"
}

You can configure as many resources as you want.

3. Initialize Terraform to install the AWS Terraform provider

If we run terraform init, we will achieve the following:

  • Initialize the backend – this will check if we are using a remote state and do the authentication to the service we are using for hosting it, or it will simply initialize the local backend (in our example, we haven’t defined any remote state, so by default, it will use the local backend).
  • Initialize the provider plugins – this will download our providers or use the binaries that we have locally.
  • It will generate a .terraform.lock.hcl file that will record our provider selections.
  • It will create a .terraform directory that will download the provider binary.

Let’s run terraform init for our configuration:

terraform init                


Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/aws versions matching "5.50.0"...
- Installing hashicorp/aws v5.50.0...
- Installed hashicorp/aws v5.50.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

We can observe that we have downloaded the exact provider version that we specified in the terraform block.

4. Apply the configuration

Now, we can run terraform apply to create the resource:

terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
 + create

Terraform will perform the following actions:

 # aws_vpc.this will be created
 + resource "aws_vpc" "this" {
     + arn                                  = (known after apply)
     + cidr_block                           = "10.0.0.0/16"
     + default_network_acl_id               = (known after apply)
     + default_route_table_id               = (known after apply)
     + default_security_group_id            = (known after apply)
     + dhcp_options_id                      = (known after apply)
     + enable_dns_hostnames                 = (known after apply)
     + enable_dns_support                   = true
     + enable_network_address_usage_metrics = (known after apply)
     + id                                   = (known after apply)
     + instance_tenancy                     = "default"
     + ipv6_association_id                  = (known after apply)
     + ipv6_cidr_block                      = (known after apply)
     + ipv6_cidr_block_network_border_group = (known after apply)
     + main_route_table_id                  = (known after apply)
     + owner_id                             = (known after apply)
     + tags_all                             = (known after apply)
   }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
 Terraform will perform the actions described above.
 Only 'yes' will be accepted to approve.

 Enter a value:

After providing “yes” to the prompt, the resource will be created:

Enter a value: yes

aws_vpc.this: Creating...
aws_vpc.this: Creation complete after 2s [id=vpc-02e183c656a74a404]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Terraform AWS provider configuration options

The most important Terraform AWS provider options include:

  • region – specify the AWS region in which we want to create your resources
  • access_key – the AWS access key we use for authenticating to the provider
  • secret_key – the AWS secret key associated with our access key
  • assume_role – configuration block for specifying what role we want to use to authenticate with AWS
  • token – session token for temporary credentials

You can look here for all the options you can configure for your AWS provider.

Authenticating to AWS with Terraform

There are many ways to authenticate using the Terraform AWS provider. Configuration for the AWS Provider can be derived from several sources, which are applied in the following order:

  1. Parameters in the provider configuration
  2. Environment variables
  3. Shared credentials and configuration files
  4. Container credentials
  5. Instance profile credentials and region
  6. External credentials process

Before we can authenticate, we will need to create an access key for use with Terraform. To do so, browse to the IAM section in the AWS console and click ‘create new access key.’

terraform aws provider - iam section
terraform aws provider - create access key

Note that the usual and recommended way to authenticate to AWS when using Terraform is via the AWS CLI, rather than any of the provider options listed above. To do this, first, install the AWS CLI, then type aws configure.

terraform aws provider - aws configure

1. Parameters in the provider configuration

provider "aws" {
  region     = "us-west-2"
  access_key = "my-access-key"
  secret_key = "my-secret-key"
}
provider "aws" {
  region = "us-west-2"
  token  = "my-token"
}

2. Environment variables

provider "aws" {}

The environment variables in the command shell are set as follows:

$ export AWS_ACCESS_KEY_ID="my-access-key"
$ export AWS_SECRET_ACCESS_KEY="my-secret-key"
$ export AWS_REGION="us-west-2"

Alternatively, you can use a token instead of Key ID and Access Key:

$ export AWS_SESSION_TOKEN="my-token"

This might be a useful option when running Terraform from a build agent in a CI/CD pipeline.

3. Shared configuration and credentials files

As mentioned in the documentation, the Terraform AWS provider can use credentials and settings from the shared configuration and credentials files.

The default paths for these files are:

  • Config
    • Linux/MacOS: ~/.aws/config
    • Windows: %USERPROFILE%\config
  • Credentials
    • Linux/MacOS: ~/.aws/credentials
    • Windows: %USERPROFILE%\credentials

If you want to use other files in the AWS Terraform provider, you will need to modify these parameters:

  • Config: shared_config_files (you can also set up the AWS_CONFIG_FILE environment variable)
  • Credentials: shared_credentials_files (you can also set up the AWS_SHARED_CREDENTIALS_FILE environment variable)

You can also change the credentials profile. By default, the credentials profile is default. If you want to change it, you can either set up the AWS_PROFILE environment variable or use the profile parameter inside of the Terraform AWS provider.

provider "aws" {
  shared_config_files      = ["path_to_config"]
  shared_credentials_files = ["path_to_credentials"]
}

4. Container credentials

The Terraform AWS provider supports IAM Task Roles for ECS and CodeBuild, as well as IAM Roles for Service Accounts (IRSA) on EKS, which enhances security and simplifies credential management when running Terraform in AWS containerized environments. 

For Task Roles, you will need to set up AWS_CONTAINER_CREDENTIALS_RELATIVE_URI and AWS_CONTAINER_CREDENTIALS_FULL_URI environment variables, which point to the location of the temporary security credentials provided to the container.

For using the Pod’s role with IRSA, the AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE environment variables are used, and they are automatically set by Kubernetes (you can also set them up manually for advanced usage).

5. Instance profile credentials and region

If you are running Terraform from an EC2 instance that has an IAM profile associated with it, it can automatically source the necessary AWS credentials directly from the EC2 Instance Metadata Service (IMDS). Both IMDS1 and the more secure IMDS2 are supported by the AWS provider.

The ec2_metadata_service_endpoint parameter in the AWS provider configuration or the AWS_EC2_METADATA_SERVICE_ENDPOINT environment variable can be used to specify a custom endpoint URL.

6. External credentials process

The Terraform AWS provider has the flexibility to also source credentials from external processes. To do that, you will need to configure the process in a named profile, which will be done in a shared credentials file.

Example shared credentials file:

[profile get_external_credentials]
credential_process = get_external_credentials --username x

Example AWS provider configuration:

provider "aws" {
  profile = "get_external_credentials"
}

Assuming IAM roles with Terraform AWS provider

provider "aws" {
  assume_role {
    role_arn     = "arn:aws:iam::123456789012:role/ROLE_NAME"
    session_name = "SESSION_NAME"
    external_id  = "EXTERNAL_ID"
  }
}

Read more about AWS IAM roles with Terraform.

Assuming an IAM role using a web identity

This approach enhances security by providing short-term credentials to Terraform. You can specify the role to assume using the role_arn parameter, and the provider will use the web identity token file added in the web_identity_token_file parameter or the AWS_WEB_IDENTITY_TOKEN_FILE environment variable.

provider "aws" {
  assume_role_with_web_identity {
    role_arn                = "arn:aws:iam::accountid:role/ROLE_NAME"
    session_name            = "SESSION_NAME"
    web_identity_token_file = "/path_to_web-identity-token"
  }
}

How to upgrade Terraform AWS provider to the newest version

If you are using your lock file that was initially generated when you ran terraform init for the first time, upgrading your version in the Terraform block and running terraform init won’t be enough:

terraform init                

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
│ Error: Failed to query available provider packages
│ Could not retrieve the list of available versions for provider hashicorp/aws: locked provider registry.terraform.io/hashicorp/aws 5.50.0 does not match
│ configured version constraint 5.51.0; must use terraform init -upgrade to allow selection of new versions

You will need to run terraform init -upgrade to upgrade to another version:

terraform init -upgrade   

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/aws versions matching "5.51.0"...
- Installing hashicorp/aws v5.51.0...
- Installed hashicorp/aws v5.51.0 (signed by HashiCorp)

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

You have now successfully upgraded your Terraform AWS provider version.

To find the latest version available, check out the Terraform docs page or the GitHub page. GitHub will also give you a version history and details on issues raised by community members and stakeholders.

Common issues and troubleshooting Terraform AWS provider

Here are a couple of common issues related to the Terraform AWS provider and how to fix them:

Issue Potential solutions
Error specifying invalid credentials S1: Check the credentials you have in your ~/.aws/credentials file and ensure they are correct.

S2: Check the values you have passed to the access keys in the environment variables.

S3: If you are using roles, check if the role exists and if you have specified it properly.

Unsupported or invalid regions S1: Check if the region you have specified exists and is enabled in your AWS account.

S2: Check the environment variable AWS_REGION and see what value you have there.

Permission denied to create or update a resource S1: Check if the user/role you are using has enough permissions to perform the action specified in the code.

 

Invalid resource configuration S1: Check your provider version. Go to the provider documentation for that specific version and add parameters accordingly.

S2: Update your provider to use the latest version.

Key points

In this article, we have shown the common uses of the Terraform AWS provider, how to authenticate using parameters in the provider configuration options, and how to use environment variables. 

We encourage you also to explore how Spacelift makes it easy to work with Terraform. If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this. It supports Git workflows, policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box. You can check it for free by creating a trial account.

Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on its existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.

Terraform Management Made Easy

Spacelift effectively manages Terraform state, more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization, and includes many more features.

Start free trial

The Practitioner’s Guide to Scaling Infrastructure as Code

Transform your IaC management to scale

securely, efficiently, and productively

into the future.

ebook global banner
Share your data and download the guide