Terraform allows IaC developers to automate the provisioning and management of several cloud resources. Leveraging a declarative syntax to describe the deployment state, it then applies it to the target environment.
One of the variables Terraform developers use to store and process values is the map variable, which stores the values in pairs, called key-value pairs.
In this post, we delve into the lookup function and explore its usage and applicability for writing IaC.
You will learn:
The Terraform lookup function is a built-in function that retrieves the values from a map variable or a list of objects. It considers the map variable’s name to retrieve the key and default values. It returns the key value only when a matching key is found. Otherwise, it returns the default value.
lookup(map name, key, default_value)Based on the above syntax, the lookup function has three primary arguments.
- Map name: A map variable that contains the key-value pairs. Read more about the Terraform map variable.
- Key: A key value that we want to retrieve from the map
- Default value: A default value that we intend to return if the map doesn’t contain the key.
Specifying the default value is optional, but best practice is to add a unique default value in the lookup function to avoid function call errors or lookup failures.
The lookup function retrieves a value from a map by its key and can provide a default value if the key is not found.
Here are some of its use cases:
1. Variable missing value
In this example, we have defined values for the instance types for the dev and prod environments, but we don’t have anything for qa. Our lookup searches for the “qa” key inside our variable, but because it doesn’t find it, it uses the default value specified “t2.small”.
variable "instance_types" {
default = {
dev = "t2.micro"
prod = "t3.large"
}
}
resource "aws_instance" "this" {
instance_type = lookup(var.instance_types, "qa", "t2.small")
}2. Conditionally creating resources
In this example, we are searching for the enable_ec2 key in our local variable. If we find it, its value will determine whether we create the instance, but if the key is not present, we will not create the instance.
For our case, because the enable_ec2 key has a true value, we will create the instance.
locals {
feature_flags = {
enable_ec2 = true
enable_s3 = false
}
}
resource "aws_instance" "this" {
count = lookup(local.feature_flags, "enable_ec2", false) ? 1 : 0
instance_type = "t2.micro"
}3. Conditionally disabling optional parameters
In the example below, we do not set any description because the description key is missing from our variable.
variable "security_group_settings" {
default = {
name = "example-sg"
}
}
resource "aws_security_group" "example" {
name = lookup(var.security_group_settings, "name", "default-sg")
description = lookup(var.security_group_settings, "description", "null")
vpc_id = "my_vpc"
}Let’s look at some examples:
Example 1. Using lookup with a map variable
Here is a sample Terraform configuration file to define a map variable “my_map”:
variable "my_map" {
type = map(string)
default = {
"key1" = "value1"
"key2" = "value2"
}
}To retrieve the value of a particular key from the above map – for example, in a local variable – use a lookup function as shown below:
locals {
my_value = lookup(var.my_map, "key1", "")
}To demonstrate this, I have created an output variable that displays the intended value when plan or apply commands.
output "my_value_output" {
value = local.my_value
}The output of the plan command below correctly displays the value of “key1.”
terraform plan
Changes to Outputs:
+ my_value_output = "value1"
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
Note that when the key does not exist in the lookup function, it returns an empty string as the default value.
Example 2. Using lookup with a nested map
In the example below, we use two lookups to reach the param1 key from the a map inside the my_nested_map. Because param1 is present there this will return 1, but if it were missing it would’ve returned 0.
locals {
my_nested_map = {
a = {
param1 = 1
param2 = 2
}
b = {
param1 = 3
param2 = 3
}
}
my_nm_lookup = lookup(lookup(local.my_nested_map, "a"), "param1", 0)
}Example 3. Using lookup on Terraform resources/data sources
The Terraform configuration below defines a security group resource and implements a local variable to look up the value of the security group ID when created. The lookup function is implemented similarly, as it is explained in the previous section.
We also define an output variable to print the value in the terminal.
resource "aws_security_group" "web" {
name = "web"
description = "Security group for web servers"
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "web"
}
}
locals {
web_sg_id = lookup(aws_security_group.web, "id", "")
}
output "my_value_output" {
value = local.web_sg_id
}Note that this value will be available in the output once the resource is created using the terraform apply command.
The output below confirms the same.
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ my_value_output = (known after apply)
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_security_group.web: Creating...
aws_security_group.web: Creation complete after 2s [id=sg-05bdcc469ccd7ddb4]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
my_value_output = "sg-05bdcc469ccd7ddb4"Alternatively, it is possible to perform a similar lookup operation on data sources included in the Terraform configuration.
Example 4. Using lookup with a list of objects
The Terraform configuration below defines a variable named “Security_Groups” to hold a list of security group objects. Each of the security group objects has these properties — name, description, and ingress configuration object.
We have provided configurations for two security groups, “SecurityGroup-Web” and “SecurityGroup-App,” in the default values.
Our intention here is to retrieve the description of “SecurityGroup-App” using the lookup function.
variable "Security_Groups" {
type = list(object(
{
name = string
description = string
ingress = optional(list(object({
from_port = number
to_port = number
protocol = string
cidr_blocks = list(string)
})), [
{
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["10.0.0.0/8"]
}])
}))
default = [
{
name = "SecurityGroups-Web"
description = "Security group for web servers"
ingress = [
{
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
},
{
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
]
},
{
name = "SecurityGroups-App"
description = "Security group for application servers"
ingress = [
{
from_port = 8081
to_port = 8081
protocol = "tcp"
cidr_blocks = ["10.0.0.0/8"]
}]
}]
}Let us define the local and output variables, as shown below.
The lookup function is implemented in the local variable “SecurityGroups_description”. The first argument implements a for loop with a conditional expression to filter the desired security group from the list.
This operation provides us with a single object from the list, which is supposed to be the first argument for the lookup function.
locals {
SecurityGroups_description = lookup(
[for sg in var.Security_Groups : sg if sg.name == "SecurityGroups-App"][0],
"description",
"")
}
output "my_value_output" {
value = local.SecurityGroups_description
}Running terraform plan on this correctly shows the description of “SecurityGroups-App”.
terraform plan
Changes to Outputs:
+ my_value_output = "Security group for application servers"
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
Notice how the ingress attribute is initialized with an “optional” modifier. In Terraform, the optional modifier declares the attribute as optionally required. If the dependent module fails to provide the value for the “ingress” attribute, Terraform automatically uses the default value provided in this optional modifier.
The default value in the optional modifier is optional. If the default value is not provided, and the dependent module also does not provide the value, then a corresponding type’s null value is automatically assigned.
Example 5: Using lookup when no key value is found
Let’s reuse the example from the nested map, and tweak it out a bit:
locals {
my_nested_map = {
a = {
param1 = 1
param2 = 2
}
b = {
param1 = 3
param2 = 3
}
}
my_nm_lookup = lookup(lookup(local.my_nested_map, "a"), "non_existent_param", 0)
}Because the “non_existent_param” is not present, we will return the default value, which in this case is 0.
Both lookup and element functions in Terraform are used for retrieving values from collections. The lookup function retrieves a value from a map by using its key, while the element function retrieves a value from a list by its index.
As you’ve seen throughout this post, the lookup function has a fallback if no value is found, whereas the element function does not support fallback values. This means that the lookup function will not return an error if the fallback is in play, but the element function will return an index out-of-range error if the index is not present.
Here are some of the best practices you should follow when working with the lookup function:
- Always provide a default value: This ensures your Terraform configuration doesn’t fail due to a missing key
- Use lookups to handle optional inputs gracefully. This enables fallback for missing attributes
- Use null as a default value in lookups to ignore attributes: This prevents setting attributes unnecessarily when they are not provided
- Avoid hardcoding keys: Use variables or dynamic expressions for keys whenever possible
- Using lookups with ternary operators: This provides more control over the fallback behavior
- Avoid overusing lookup: If a key is always present, access it directly instead of using lookups
Terraform is really powerful, but to achieve an end-to-end secure Gitops approach, you need to use a product that can run your Terraform workflows. Spacelift takes managing Terraform to the next level by giving you access to a powerful CI/CD workflow and unlocking features such as:
- Policies (based on Open Policy Agent) – You can control how many approvals you need for runs, what kind of resources you can create, and what kind of parameters these resources can have, and you can also control the behavior when a pull request is open or merged.
- Multi-IaC workflows – Combine Terraform with Kubernetes, Ansible, and other infrastructure-as-code (IaC) tools such as OpenTofu, Pulumi, and CloudFormation, create dependencies among them, and share outputs
- Build self-service infrastructure – You can use Blueprints to build self-service infrastructure; simply complete a form to provision infrastructure based on Terraform and other supported tools.
- Integrations with any third-party tools – You can integrate with your favorite third-party tools and even build policies for them. For example, see how to Integrate security tools in your workflows using Custom Inputs.
Spacelift enables you to create private workers inside your infrastructure, which helps you execute Spacelift-related workflows on your end. For more information on configuring private workers, refer to the documentation.
You can check it for free by creating a trial account or booking a demo with one of our engineers.
The Terraform lookup function simplifies data management and retrieves values from a map variable or a list of objects. The function has three arguments: map, key, and default value. You can use the function to retrieve the value of a specific key or from a list of objects.
The lookup function also handles the key-value pairs efficiently. Finally, you can use the lookup function in different scenarios to simplify the infrastructure configurations defined using Terraform HCL.
Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.
Automate Terraform Deployments with Spacelift
Automate your infrastructure provisioning, and build more complex workflows based on Terraform using policy as code, programmatic configuration, context sharing, drift detection, resource visualization, and many more.
