In this article, we will give an overview of what AWS provider is in Terraform and show everything from basic concepts to advanced configuration and authentication methods for managing AWS resources using Terraform.
What we will cover:
- What is a Terraform provider?
- What is Terraform AWS provider?
- How to set up the Terraform AWS provider?
- Terraform AWS provider configuration options
- Authenticating to AWS with Terraform
- Assuming IAM roles with Terraform AWS provider
- How to upgrade Terraform AWS provider to the newest version?
- Terraform AWS provider – Common issues and troubleshooting
A provider in Terraform is a plugin that enables communication with external APIs and services. It lets Terraform interact with various cloud platforms, such as AWS, infrastructure providers, and other third-party services. Providers are specified within the Terraform configuration code, allowing users to define the specific services and resources they need to manage and provision.
For more details on Terraform providers (including the Spacelift Terraform provider), check out our previous article: Terraform Providers Overview.
The Terraform AWS provider is a plugin that enables seamless integration between Terraform and the Amazon Web Services (AWS) cloud platform. It allows users to define, provision, and manage AWS resources using Terraform’s declarative configuration language. Before using this provider, you must configure it with the appropriate credentials to authenticate and authorize access to your AWS account.
Key features of the Terraform AWS provider
As for the majority of Terraform providers, the Terraform AWS provider comes with the following key features:
- Resource management – create/modify/delete AWS resources based on your Terraform configuration, such as EC2 instances, VPCs, load balancers, S3 buckets, and others
- Data sources – get information about existing AWS resources
- State management with S3 and Dynamo DB – you can use AWS for remote state management using S3 to host the state and Dynamo DB for locking (Read more: How to Manage DynamoDB Tables With Terraform)
- Great modules community – Because AWS is the leader cloud provider, it also has the biggest modules community
Let’s see the steps required to have the AWS provider up and running. The process for using the Terraform AWS provider is the following:
- Provider configuration
- AWS resource configuration
- Initializing Terraform to install the provider
- Running Terraform apply to create the resources
1. Configure the provider
You can configure your AWS provider in many ways listed here.
Because Terraform can read information from your environment if you have set up your AWS credentials, you could configure the provider just by specifying the region:
provider "aws" {
region = "eu-west-1"
}If we don’t specify a Terraform block with version constraint for the provider, Terraform will automatically download the latest available version of the AWS provider.
At the time of writing, the latest version of the AWS provider is 5.51.1. Let’s add a Terraform block that makes us use the 5.50.0 version:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "5.50.0"
}
}
}2. Define the AWS resources
Based on the provider above, we can configure AWS resources. For this example, we will just configure an AWS VPC:
resource "aws_vpc" "this" {
cidr_block = "10.0.0.0/16"
}You can configure as many resources as you want.
3. Initialize Terraform to install the AWS Terraform provider
If we run terraform init, we will achieve the following:
- Initialize the backend – this will check if we are using a remote state and do the authentication to the service we are using for hosting it, or it will simply initialize the local backend (in our example, we haven’t defined any remote state, so by default, it will use the local backend).
- Initialize the provider plugins – this will download our providers or use the binaries that we have locally.
- It will generate a .terraform.lock.hcl file that will record our provider selections.
- It will create a .terraform directory that will download the provider binary.
Let’s run terraform init for our configuration:
terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "5.50.0"...
- Installing hashicorp/aws v5.50.0...
- Installed hashicorp/aws v5.50.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
We can observe that we have downloaded the exact provider version that we specified in the terraform block.
4. Apply the configuration
Now, we can run terraform apply to create the resource:
terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_vpc.this will be created
+ resource "aws_vpc" "this" {
+ arn = (known after apply)
+ cidr_block = "10.0.0.0/16"
+ default_network_acl_id = (known after apply)
+ default_route_table_id = (known after apply)
+ default_security_group_id = (known after apply)
+ dhcp_options_id = (known after apply)
+ enable_dns_hostnames = (known after apply)
+ enable_dns_support = true
+ enable_network_address_usage_metrics = (known after apply)
+ id = (known after apply)
+ instance_tenancy = "default"
+ ipv6_association_id = (known after apply)
+ ipv6_cidr_block = (known after apply)
+ ipv6_cidr_block_network_border_group = (known after apply)
+ main_route_table_id = (known after apply)
+ owner_id = (known after apply)
+ tags_all = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value:
After providing “yes” to the prompt, the resource will be created:
Enter a value: yes
aws_vpc.this: Creating...
aws_vpc.this: Creation complete after 2s [id=vpc-02e183c656a74a404]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.The most important Terraform AWS provider options include:
- region – specify the AWS region in which we want to create your resources
- access_key – the AWS access key we use for authenticating to the provider
- secret_key – the AWS secret key associated with our access key
- assume_role – configuration block for specifying what role we want to use to authenticate with AWS
- token – session token for temporary credentials
You can look here for all the options you can configure for your AWS provider.
There are many ways to authenticate using the Terraform AWS provider. Configuration for the AWS Provider can be derived from several sources, which are applied in the following order:
- Parameters in the provider configuration
- Environment variables
- Shared credentials and configuration files
- Container credentials
- Instance profile credentials and region
- External credentials process
Before we can authenticate, we will need to create an access key for use with Terraform. To do so, browse to the IAM section in the AWS console and click ‘create new access key.’
Note that the usual and recommended way to authenticate to AWS when using Terraform is via the AWS CLI, rather than any of the provider options listed above. To do this, first, install the AWS CLI, then type aws configure.
You can then enter your access key ID, secret access key, and default region.
1. Parameters in the provider configuration
To specify parameters in the AWS provider configuration, we can set an access key and secret key as follows:
provider "aws" {
region = "us-west-2"
access_key = "my-access-key"
secret_key = "my-secret-key"
}Note: This is NOT recommended! If your secrets are hardcoded into your configuration files and committed to source control, they may be compromised.
Similarly, we can specify a session access token, typically provided after a successful identity federation or Multi-Factor Authentication (MFA) login. With MFA login, this is the session token provided afterward, not the 6-digit MFA code used to get temporary credentials.
provider "aws" {
region = "us-west-2"
token = "my-token"
}2. Environment variables
To use the environment variables option to authenticate AWS with Terraform, you can provide the credentials using the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and optionally AWS_SESSION_TOKEN environment variables. The region can be set using the AWS_REGION or AWS_DEFAULT_REGION environment variables.
In this case, the provider configuration options block would be empty, as the credentials needed for authentication are supplied at the system level (i.e., these are local to the system you are running Terraform from). This is safer than hardcoding your secrets and tokens in the configuration files.
provider "aws" {}The environment variables in the command shell are set as follows:
$ export AWS_ACCESS_KEY_ID="my-access-key"
$ export AWS_SECRET_ACCESS_KEY="my-secret-key"
$ export AWS_REGION="us-west-2"Alternatively, you can use a token instead of Key ID and Access Key:
$ export AWS_SESSION_TOKEN="my-token"This might be a useful option when running Terraform from a build agent in a CI/CD pipeline.
3. Shared configuration and credentials files
As mentioned in the documentation, the Terraform AWS provider can use credentials and settings from the shared configuration and credentials files.
The default paths for these files are:
- Config
- Linux/MacOS: ~/.aws/config
- Windows: %USERPROFILE%\config
- Credentials
- Linux/MacOS: ~/.aws/credentials
- Windows: %USERPROFILE%\credentials
If you want to use other files in the AWS Terraform provider, you will need to modify these parameters:
- Config: shared_config_files (you can also set up the AWS_CONFIG_FILE environment variable)
- Credentials: shared_credentials_files (you can also set up the AWS_SHARED_CREDENTIALS_FILE environment variable)
You can also change the credentials profile. By default, the credentials profile is default. If you want to change it, you can either set up the AWS_PROFILE environment variable or use the profile parameter inside of the Terraform AWS provider.
provider "aws" {
shared_config_files = ["path_to_config"]
shared_credentials_files = ["path_to_credentials"]
}
4. Container credentials
The Terraform AWS provider supports IAM Task Roles for ECS and CodeBuild, as well as IAM Roles for Service Accounts (IRSA) on EKS, which enhances security and simplifies credential management when running Terraform in AWS containerized environments.
For Task Roles, you will need to set up AWS_CONTAINER_CREDENTIALS_RELATIVE_URI and AWS_CONTAINER_CREDENTIALS_FULL_URI environment variables, which point to the location of the temporary security credentials provided to the container.
For using the Pod’s role with IRSA, the AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE environment variables are used, and they are automatically set by Kubernetes (you can also set them up manually for advanced usage).
5. Instance profile credentials and region
If you are running Terraform from an EC2 instance that has an IAM profile associated with it, it can automatically source the necessary AWS credentials directly from the EC2 Instance Metadata Service (IMDS). Both IMDS1 and the more secure IMDS2 are supported by the AWS provider.
The ec2_metadata_service_endpoint parameter in the AWS provider configuration or the AWS_EC2_METADATA_SERVICE_ENDPOINT environment variable can be used to specify a custom endpoint URL.
6. External credentials process
The Terraform AWS provider has the flexibility to also source credentials from external processes. To do that, you will need to configure the process in a named profile, which will be done in a shared credentials file.
Example shared credentials file:
[profile get_external_credentials]
credential_process = get_external_credentials --username xExample AWS provider configuration:
provider "aws" {
profile = "get_external_credentials"
}Another useful way of using the AWS Provider options is the ability to assume an IAM role. This is done using the role_arn option inside the assume_role block.
provider "aws" {
assume_role {
role_arn = "arn:aws:iam::123456789012:role/ROLE_NAME"
session_name = "SESSION_NAME"
external_id = "EXTERNAL_ID"
}
}Read more about AWS IAM roles with Terraform.
Assuming an IAM role using a web identity
This approach enhances security by providing short-term credentials to Terraform. You can specify the role to assume using the role_arn parameter, and the provider will use the web identity token file added in the web_identity_token_file parameter or the AWS_WEB_IDENTITY_TOKEN_FILE environment variable.
provider "aws" {
assume_role_with_web_identity {
role_arn = "arn:aws:iam::accountid:role/ROLE_NAME"
session_name = "SESSION_NAME"
web_identity_token_file = "/path_to_web-identity-token"
}
}If you are using your lock file that was initially generated when you ran terraform init for the first time, upgrading your version in the Terraform block and running terraform init won’t be enough:
terraform init
Initializing the backend...
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
╷
│ Error: Failed to query available provider packages
│
│ Could not retrieve the list of available versions for provider hashicorp/aws: locked provider registry.terraform.io/hashicorp/aws 5.50.0 does not match
│ configured version constraint 5.51.0; must use terraform init -upgrade to allow selection of new versionsYou will need to run terraform init -upgrade to upgrade to another version:
terraform init -upgrade
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "5.51.0"...
- Installing hashicorp/aws v5.51.0...
- Installed hashicorp/aws v5.51.0 (signed by HashiCorp)
Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
You have now successfully upgraded your Terraform AWS provider version.
To find the latest version available, check out the Terraform docs page or the GitHub page. GitHub will also give you a version history and details on issues raised by community members and stakeholders.
Here are a couple of common issues related to the Terraform AWS provider and how to fix them:
| Issue | Potential solutions |
| Error specifying invalid credentials | S1: Check the credentials you have in your ~/.aws/credentials file and ensure they are correct.
S2: Check the values you have passed to the access keys in the environment variables. S3: If you are using roles, check if the role exists and if you have specified it properly. |
| Unsupported or invalid regions | S1: Check if the region you have specified exists and is enabled in your AWS account.
S2: Check the environment variable AWS_REGION and see what value you have there. |
| Permission denied to create or update a resource | S1: Check if the user/role you are using has enough permissions to perform the action specified in the code.
|
| Invalid resource configuration | S1: Check your provider version. Go to the provider documentation for that specific version and add parameters accordingly.
S2: Update your provider to use the latest version. |
In this article, we have shown the common uses of the Terraform AWS provider, how to authenticate using parameters in the provider configuration options, and how to use environment variables.
We encourage you also to explore how Spacelift makes it easy to work with Terraform. If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this. It supports Git workflows, policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box. You can check it for free by creating a trial account.
Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on its existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.
Terraform Management Made Easy
Spacelift effectively manages Terraform state, more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization, and includes many more features.
