Terraform Regex – Regular Expressions with Examples

In this article, we will dive into the wonderful world of Regular expressions (Regex or Regexp), first explaining what they are and why you might need to use them. We will show some examples of how to use these expressions with the regex and regexall functions in your Terraform code, and we will also take a look at the replace function. 

What we will cover:

1. What are regular expressions?
2. What regex does Terraform use?
3. What is the regex function in Terraform?
4. Example: Using the Terraform regex function
5. What is the regexall function in Terraform?
6. Example: Using the Terraform regexall function
7. What is the replace string function in Terraform?
8. Example: Using the Terraform replace function

Regex expressions are special text patterns used to match specific patterns within text. They are a powerful tool for searching, extracting, and manipulating text data in various applications, such as text editors, search tools, and programming languages. 

Regular expressions are supported in many programming languages, often through built-in libraries or modules. The concept of regular expressions dates back to the 1950s when mathematician Stephen Cole Kleene formalized the idea of regular languages.

The common use cases for regex include:

• Text search and manipulation – Finding specific words or phrases within a large document
• Data extraction – Extracting specific data points from text, such as phone numbers or email addresses, from a log file
• Input validation – Ensuring user input conforms to a specific format, like a valid email address or password
• Code search and refactoring – Finding specific code patterns within large codebases

Terraform uses Google’s RE2 regular expression engine, which is known for its high efficiency and comprehensive feature set. However, although Terraform offers a wide array of regex functionalities, it does not encompass the full spectrum of capabilities available in the RE2 engine.

One notable limitation is the restricted support for backreferences. Backreferences allow you to reference previously matched parts of the text within the expression, but Terraform’s implementation has restrictions. It supports:

• Character-matching (literal characters and special characters)
• Repetition (specifying how many times a pattern can occur)
• Alternatives (matching one of several options)
• Grouping and capturing subpatterns
• Backreferences (referring to previously captured subpatterns) (with limited support in Terraform)

You can find documentation on RE2 on GitHub. As Regex expressions can quickly become complex, you can use an online test tool that includes the RE2 engine to validate your Regex expression.

Regex patterns

We’ll break down these regex patterns into the key concepts you need to understand to use regular expressions effectively. You can use the reference below to help you form your own regex expressions.

Common patterns quick reference

• [a-z]: Matches any lowercase letter
• \d+: Matches one or more digits
• ^abc$: Matches the exact string “abc” (beginning and end of line)
• .*: Matches any sequence of characters (zero or more)

Literals

Literal characters match themselves. For example, the regex abc matches the string “abc”.

Metacharacters

Metacharacters are the special characters with specific meanings in regex. Common metacharacters include:

• .: Matches any single character except newline
• ^: Matches the start of a string
• $: Matches the end of a string
• *: Matches 0 or more occurrences of the preceding element
• +: Matches 1 or more occurrences of the preceding element
• ?: Matches 0 or 1 occurrence of the preceding element
• []: Matches any single character within the brackets (character class)
• |: Acts as a logical OR operator
• (): Groups patterns and captures the matched text

Character classes

• [abc]: Matches any one of the characters a, b, or c
• [a-z]: Matches any lowercase letter from a to z
• [0-9]: Matches any digit from 0 to 9
• [^abc]: Matches any character except a, b, or c

Quantifiers

• *: Matches 0 or more occurrences
• +: Matches 1 or more occurrences
• ?: Matches 0 or 1 occurrence
• {n}: Matches exactly n occurrences
• {n,}: Matches n or more occurrences
• {n,m}: Matches between n and m occurrences

Anchors

• ^: Matches the start of a string
• $: Matches the end of a string
• \b: Matches a word boundary
• \B: Matches a non-word boundary

Escape sequences

• \d: Matches any digit (equivalent to [0-9])
• \D: Matches any non-digit
• \w: Matches any word character (alphanumeric plus underscore)
• \W: Matches any non-word character
• \s: Matches any whitespace character
• \S: Matches any non-whitespace character

The regex function in Terraform applies a regular expression pattern to a string and returns the matched substrings. It returns a list of the matches and, if no match is found, it returns an empty list.

The syntax is as follows:

Where

• string – The text string you want to search against
• pattern – The regular expression pattern that defines what you want to match within the string

Let’s see some examples using the Terraform regex function.

Example 1: Matching a domain name

In this example, we define a variable website_url that might hold a URL like “https://www.example.com”.

• The regex function extracts the domain name “example.com” using a pattern that matches any characters followed by a dot (“.”) and then again any characters (but not another dot).
• The element function is used to access the first element (index 0) of the returned list, which contains the captured domain name.

Example 2: Validating input variables

You can also use the regex function to validate input variables. For instance, to ensure that a variable follows a specific format:

In this example, local.email_valid will be true if var.email is a valid email address according to the regex pattern.

Suppose you want to determine whether a string starts with the letter ‘p’ followed by one or more digits:

• If my_input is “pab1”, it evaluates to “testing”.
• If my_input is “p123”, it evaluates to “production”.

The regexall function in Terraform is a variant of the regex function specifically designed to find all potential matches of a regular expression pattern within a string. The function returns a list, even if there’s only one match or no matches at all, in which case it returns an empty list.

The syntax for this function is as follows:

Where:

• string: The text string you want to search against.
• pattern: The regular expression pattern defining what you want to match.

What is the difference between regex and regexall in Terraform?

Both regex and regexall functions in Terraform work with capture groups in regular expressions. However, regexall returns a list of captured elements for each match, whereas regex might return a single list or map depending on the capture group structure. 

If the pattern matches multiple times within the string, regexall captures all occurrences and returns them as a list. regexall is specifically intended to find all matching substrings, whereas regex typically returns only the first match (or the entire matched string if no capture groups are used).

Use regexall when you need to find all occurrences of a pattern within a string, and use the regex function when you’re only interested in the first match or want to perform specific actions based on the existence of a single match (e.g., validating a format).

Now, let’s go explore some examples using the regexall function.

Example 1: Finding all email addresses

In this example, we are searching for all email addresses within a text block stored in text_data.

The regexall function here returns a list containing all email addresses found in the text.

Example 2: Extracting tags with named capture groups

In the next example, we parse a log line to extract all key-value pairs representing tags.

The regex pattern uses named capture groups: (?P<key>[^ ]+) for the tag key and (?P<value>.*) for the value. regexall returns a list of maps, where each map represents a single tag with its key and value.

The replace function performs a simple string substitution based on a provided pattern.  It does not support regular expressions for pattern matching, but it can be useful when you need to modify strings dynamically within your Terraform configurations.

Syntax:

Where: 

• string: The text string where you want to perform the replacement.
• old: The substring you want to replace.
• new: The replacement string that will take the place of old.

Example 1

In this first example, we conditionally define a new server name based on the environment.

We want the replace function to replace all occurrences of “dev” with the value of new_name within the old_name string.

Example 2

Here, the replace("Hello", "Hi", var.input_string) replaces the substring “Hello” with “Hi” in var.input_string, and we want the local.replaced_string to contain “Hi, World!” as the output.

Terraform is really powerful, but to achieve an end-to-end secure Gitops approach, you need to use a product that can run your Terraform workflows. Spacelift takes managing Terraform to the next level by giving you access to a powerful CI/CD workflow and unlocking features such as:

• Policies (based on Open Policy Agent) – You can control how many approvals you need for runs, what kind of resources you can create, and what kind of parameters these resources can have, and you can also control the behavior when a pull request is open or merged.
• Multi-IaC workflows – Combine Terraform with Kubernetes, Ansible, and other infrastructure-as-code (IaC) tools such as OpenTofu, Pulumi, and CloudFormation,  create dependencies among them, and share outputs
• Build self-service infrastructure – You can use Blueprints to build self-service infrastructure; simply complete a form to provision infrastructure based on Terraform and other supported tools.
• Integrations with any third-party tools – You can integrate with your favorite third-party tools and even build policies for them. For example, see how to Integrate security tools in your workflows using Custom Inputs.

Spacelift enables you to create private workers inside your infrastructure, which helps you execute Spacelift-related workflows on your end. For more information on configuring private workers, refer to the documentation.

You can check it for free by creating a trial account or booking a demo with one of our engineers.

Regular expressions can be used within your terraform code anywhere pattern matching is required.

• Use regexall when you need to find all occurrences of a pattern within a string. 
• Use regex when you’re only interested in the first match or want to perform specific actions based on the existence of a single match (e.g., validating a format).
• Use replace for simple string replacements without the need for regular expressions.

Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.