In this article, we will give an overview of what providers are in Terraform, and show how to use the provider for Amazon Web Services (AWS) with some useful examples.
A provider in Terraform is a plugin that enables interaction with an API. This includes Cloud providers such as AWS. The providers are specified in the Terraform configuration code, telling Terraform which services it needs to interact with.
For more details on Terraform providers (including the Spacelift Terraform provider), check out our previous article: Terraform Providers Overview.
Note: New versions of Terraform will be placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that will expand on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6. OpenTofu retained all the features and functionalities that had made Terraform popular among developers while also introducing improvements and enhancements. OpenTofu is not going to have its own providers and modules, but it is going to use its own registry for them.
To use Terraform to manage and deploy resources and infrastructure to AWS, you will need to use the AWS provider. You must configure the provider with the proper credentials before you can use it. This provider is maintained internally by the HashiCorp AWS Provider team.
To install the AWS provider, the example configuration below can be used (usually in your main.tf file):
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "4.27.0"
}
}
}
provider "aws" {
# Configuration options
}Note that the version of the provider is ‘pinned’ here to “4.27.0” (the latest at the time of writing). This is recommended best practice to avoid any unexpected changes in behavior between provider versions. Newer minor versions should be backward compatible and not introduce issues, however, major version updates may introduce breaking changes. Pinning the version allows you to update manually when you are confident there will be no adverse effects in doing so.
To find the latest version available, check out the Terraform docs page, or the GitHub page. GitHub will also give you a version history and details on issues raised by community members and stakeholders.
The configuration options that can be specified in the provider block are all optional for the AWS provider. As well as the general options available for all Terraform providers such as aliasand version, these provider-specific options can be used to instruct Terraform on how to interact with AWS. For example, how to authenticate to your AWS subscription, specify the region, or assume an IAM role.
From the Terraform docs, there are a number of ways to authenticate using the AWS provider. Configuration for the AWS Provider can be derived from several sources, which are applied in the following order:
- Parameters in the provider configuration
- Environment variables
- Shared credentials and configuration files
- Container credentials
- Instance profile credentials and region
- Using an external credentials process
We will demonstrate the most common methods of using parameters in the provider configuration and environment variables.
However, before we can authenticate, we will need to create an access key for use with Terraform. Browse to the IAM section in the AWS console and ‘create new access key’.
Note that the usual and recommended way to authenticate to AWS when using Terraform is via the AWS CLI, rather than any of the provider options listed above. To do this, first, install the AWS CLI, then type aws configure.
You can then enter your access key ID, secret access key, and default region.
Parameters in the provider configuration
To specify parameters in the provider configuration, we can set an access key and secret key as follows:
provider "aws" {
region = "us-west-2"
access_key = "my-access-key"
secret_key = "my-secret-key"
}Note: This is NOT recommended! If your secrets are hardcoded into your configuration files and committed to source control, they may be compromised.
Similarly, we can specify a session access token, typically provided after a successful identity federation or Multi-Factor Authentication (MFA) login. With MFA login, this is the session token provided afterward, not the 6-digit MFA code used to get temporary credentials.
provider "aws" {
region = "us-west-2"
token = "my-token"
}Environment Variables
To use the environment variables option to authenticate, credentials can be provided by using the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and optionally AWS_SESSION_TOKEN environment variables. The region can be set using the AWS_REGION or AWS_DEFAULT_REGION environment variables.
In this case, the provider configuration options block would be empty, as the credentials needed for authentication are supplied at the system level (i.e., these are local to the system you are running Terraform from). This is safer than hardcoding your secrets and tokens in the configuration files.
provider "aws" {}In the command shell, the environment variables are set as follows:
$ export AWS_ACCESS_KEY_ID="my-access-key"
$ export AWS_SECRET_ACCESS_KEY="my-secret-key"
$ export AWS_REGION="us-west-2"Alternatively, a token can be used instead of Key ID and Access Key:
$ export AWS_SESSION_TOKEN="my-token"This might be a useful option when running Terraform from a build agent in a CI/CD pipeline.
Shared Configuration and Credentials Files
The AWS provider can use credentials and settings from the shared configuration and credentials files as mentioned in their documentation.
The default paths for these files are:
- Config
- Linux/MacOS: ~/.aws/config
- Windows: %USERPROFILE%\config
- Credentials
- Linux/MacOS: ~/.aws/credentials
- Windows: %USERPROFILE%\credentials
If you want to use other files in the AWS Terraform provider, you will need to modify these parameters:
- Config: shared_config_files (you can also set up the AWS_CONFIG_FILE environment variable)
- Credentials: shared_credentials_files (you can also set up the AWS_SHARED_CREDENTIALS_FILE environment variable)
You can also change the credentials profile. By default, the credentials profile is default, but if you want to change it, you can either set up the AWS_PROFILE environment variable or use the profile parameter inside of the Terraform AWS provider.
provider "aws" {
shared_config_files = ["path_to_config"]
shared_credentials_files = ["path_to_credentials"]
}
Container Credentials
The Terraform AWS provider supports IAM Task Roles for ECS and CodeBuild, as well as IAM Roles for Service Accounts (IRSA) on EKS, which enhances security and simplifies credential management when running Terraform in AWS containerized environments.
For Task Roles, you will need to set up AWS_CONTAINER_CREDENTIALS_RELATIVE_URI and AWS_CONTAINER_CREDENTIALS_FULL_URI environment variables, which point to the location of the temporary security credentials provided to the container.
For using the Pod’s role with IRSA, the AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE environment variables are used, and they are automatically set by Kubernetes (you can also set them up manually for advanced usage).
Instance profile credentials and region
If you are running Terraform from an EC2 instance that has an IAM profile associated with it, it can automatically source the necessary AWS credentials directly from the EC2 Instance Metadata Service (IMDS). Both IMDS1 and the more secure IMDS2 are supported by the AWS provider.
The ec2_metadata_service_endpoint parameter in the AWS provider configuration or the AWS_EC2_METADATA_SERVICE_ENDPOINT environment variable can be used to specify a custom endpoint URL.
Assume an IAM role
Another useful use of the AWS Provider options is the ability to assume an IAM role. This is done using the role_arn option inside the assume_role block.
provider "aws" {
assume_role {
role_arn = "arn:aws:iam::123456789012:role/ROLE_NAME"
session_name = "SESSION_NAME"
external_id = "EXTERNAL_ID"
}
}Read more about AWS IAM roles with Terraform.
Assuming an IAM Role Using A Web Identity
This approach enhances security by providing short-term credentials to Terraform. You can specify the role to assume using the role_arn parameter, and the provider will use the web identity token file added in the web_identity_token_file parameter or the AWS_WEB_IDENTITY_TOKEN_FILE environment variable.
provider "aws" {
assume_role_with_web_identity {
role_arn = "arn:aws:iam::accountid:role/ROLE_NAME"
session_name = "SESSION_NAME"
web_identity_token_file = "/path_to_web-identity-token"
}
}Using an External Credentials Process
You even have the flexibility to source credentials from external processes. To do that, you will need to configure the process in a named profile, and this configuration will take place in a shared credentials file.
Example shared credentials file:
[profile get_external_credentials]
credential_process = get_external_credentials --username xExample AWS provider configuration:
provider "aws" {
profile = "get_external_credentials"
}In this article, we have shown the common uses of the Terraform AWS provider, showing how to authenticate using parameters in the provider configuration options, and using environment variables. For a full list of available options, check out the Terraform docs page.
We encourage you also to explore how Spacelift makes it easy to work with Terraform. If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this. It supports Git workflows, policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box. You can check it for free by creating a trial account.
Terraform Management Made Easy
Spacelift effectively manages Terraform state, more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization, and includes many more features.
Terraform CLI Commands Cheatsheet
Initialize/ plan/ apply your IaC, manage modules, state, and more.
