Terraform Fileset Function: Filter & Deploy Specific Files

The fileset function in Terraform returns a set of file paths matching a given glob pattern in a directory. It’s typically used to load multiple files for processing, such as templates or static assets. Returned paths are relative to the base directory and always use / as the path separator, regardless of OS.

This function takes two arguments: the base directory and a glob pattern (e.g., "*.tf" or "**/*.yaml"). The result is an unordered set of relative paths. If you need a deterministic sequence, wrap it with sort(tolist(...)).

For example:

This example returns all .tf files in the specified directory, excluding subdirectories. You can pair it with for_each or loops to read or template files in batch. 

Let’s consider some examples.

Example 1: Listing all .tf files in a directory

To list all .tf files in a directory using the Terraform fileset function, use the following syntax:

This expression returns a set of .tf file names (not full paths) within the current module directory. The fileset function matches files using a glob pattern, so "*.tf" will include all Terraform files at the top level of the specified path. 

To include .tf files recursively in subdirectories, use:

Terraform supports the recursive ** (globstar) when it is its own path component (as above). The returned set contains relative paths from the root of path.module. This is useful for templating, validation logic, or module introspection in custom Terraform configurations.

Example 2: Using fileset() with for_each to create resources

You can use fileset() with for_each in Terraform to dynamically create resources based on files matching a pattern. fileset() returns a list of file paths relative to a given directory, which can be converted to a map for use with for_each.

For example, suppose you have a directory called data/ that contains several .json files. You can collect these files using fileset() and prepare them for for_each like this:

This creates a map where each key is a filename (e.g., policy1.json) and each value is its full path (e.g., ./data/policy1.json). You can then use for_each in a resource block:

This creates one IAM policy per file, using the filename (minus the extension) as the resource name and the file contents as the policy document.

Example 3: Combining fileset() with templatefile() to render templates

You can combine fileset() and templatefile() in Terraform to programmatically render multiple templates with shared input variables. This is especially useful when managing a set of configuration files that follow a consistent format, such as service definitions, cloud-init scripts, or Kubernetes manifests.

Read more: What are Terraform Templates? Examples and Use Cases

Start by organizing your templates in a folder. For example, suppose you have a templates/ directory with several .tpl files:

Each .tpl file might contain references to variables like ${variable1} or ${variable2}.

In your Terraform configuration, use the following code to locate and render each template:

Here’s what each part does:

• fileset("${path.module}/templates", "*.tpl") finds all .tpl files in the templates/ directory.
• templatefile() renders each template with a provided map of variables. You can call any Terraform function inside the template. However, recursive calls to templatefile() are not permitted, and variable names must start with a letter.
• The for expression builds a map where the key is the filename and the value is the rendered string.

You can later use local.rendered_templates in other parts of your Terraform configuration, such as passing rendered templates into a resource or writing them to local files. This pattern scales efficiently when dealing with multiple configuration files.

Example 4: Filtering and deploying only specific file types

To filter and deploy only specific file types using the Terraform fileset function, you can define a glob pattern that matches the desired extensions. 

For example, to include only .yaml files, use:

The fileset function returns a set of file names (relative to the specified directory) matching the pattern. This allows precise control over which files are included in a deployment by filtering based on file type or naming convention.

To deploy these files using a resource like kubernetes_manifest, loop over the filtered set using for_each, decoding YAML to an HCL object:

kubernetes_manifest requires an HCL object and yamldecode(file(...)) is the standard way to feed YAML manifests.

If you need multiple extensions, prefer setunion over flatten:

setunion produces a single deduplicated set.

Terraform’s fileset function is handy for dynamically gathering lists of files matching a pattern within a given directory. However, when you begin using it with complex file structures or try to match files deeply nested in subdirectories, things can get tricky.

Let’s walk through the most common issues developers face and how to address them.

1. Misunderstanding pattern behavior

One of the most frequent mistakes is assuming that fileset supports recursive glob patterns like **/*.tf, similar to what you might use in a Unix shell or tools like rsync.

This does work in Terraform — ** is supported for recursive matching when it is its own path component. For example, **/*.tf matches at any depth, but path** is not valid; use path*/** instead.

If you want to match a specific filename at any depth:

2. Confusing relative vs. absolute paths

Another surprise is that fileset returns relative paths, not absolute or full paths. This often breaks later operations if you try to use these paths directly in places like file functions or rendering logic.

For example:

To use these file paths correctly, prepend the base directory:

Now local.full_paths contains valid paths like modules/foo/main.tf, which can be used safely. Terraform also normalizes path separators to / across OSes, and functions like file() run during configuration evaluation, so the files must exist at plan time.

3. Incorrect depth in patterns

Since Terraform doesn’t support recursive globs, users sometimes try to match deeper files with something like:

But if you don’t know the depth in advance, use the recursive globstar:

4. Ignoring empty results

When no files match your pattern, fileset simply returns an empty set. If you don’t check for this, you might run into errors when referencing elements or looping over the results.

If you need positional indexing or sorting, convert with tolist() first.

5. Assuming stable ordering

The files returned by fileset are not guaranteed to be in any specific order. This can cause unintended issues if you’re relying on consistent ordering across runs or environments.

To avoid this, explicitly sort the list:

6. Lack of filtering capabilities

fileset only supports filename pattern matching. It doesn’t let you filter based on file content, metadata, or even extensions in a precise way.

If you need more advanced filtering (e.g., only include .tf files that contain a certain string), you’d need to combine fileset with additional filtering logic:

This example filters out files that don’t contain the word provider. For pattern power, globs also support ?, character classes like [a-z], and alternation with {a,b}, in addition to * and **.

In summary, Terraform’s fileset function offers a simple way to match files by pattern dynamically, streamlining automation and reducing manual updates. Used wisely, it improves flexibility, scalability, and maintainability in your infrastructure as code workflows.