Debugging and logging are the lifeline of developers when it comes to troubleshooting any kind of issue. If something has failed in a production environment, developers always look for the application logs which can give them the type, cause, and time of the error.
The exact same principle is also followed when debugging the Terraform script. Terraform projects can grow very quickly when worked on by multiple teams. If you do not have a proper logging setup for troubleshooting, the project team will have a very difficult time troubleshooting and debugging.
In this article, we will focus on the commonly followed practices for debugging Terraform Projects.
How to enable the logs for debugging? How to set the log file for debugging? How to manage log levels? Here’s how.
The only prerequisite we have for this blog post is you must install Terraform on your host machine.
You can verify your Terraform installation by running the following Terraform version command:
$ terraform -version
Terraform v1.1.0
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v3.69.0There are certain log levels provided by Terraform for debugging and troubleshooting. As developers, we have to choose and set the log level for our Terraform project.
Types of Log Level
In total, there 5 log levels which can be used for debugging purposes:
- TRACE—one of the most descriptive log levels, if you set the log level to TRACE, Terraform will write every action and step into the log file.
- DEBUG—a little bit more sophisticated logging which is used by developers at critical or more complex pieces of code to reduce debugging time.
-
- INFO—the info log level is useful when needing to log some informative instructions or readme type instructions.
- WARN—used when something is not critical but would be nice to include in the form of a log so that the developer can make adjustments later.
- ERROR—as the name suggests, this is used if something is terribly wrong and is a blocker.
How to set the log level to enable debugging?
The log level can be set with the environment variable TF_LOG. You need to export the variable with the correct log level. Here is an example of where TF_LOG is set with DEBUG level:
$ export TF_LOG=”DEBUG”Verify the log level:
$ echo $TF_LOG
DEBUG
After setting the log level, the next thing we need to set is the Log File Path, and for that we are going to use the environment variable TF_LOG_PATH.
For this blog post we are going to create a log file named terraform-debug.log.
We are going to place this file under the directory /home/vagrant/terraform-ec2-aws.
*Note – You can name and choose the log file path as per your requirement.
Let’s export the TF_LOG_PATH
export TF_LOG_PATH="/home/vagrant/terraform-ec2-aws/terraform-debug.log"Now, after setting both Log Level and Log File path, let’s create a very small Terraform project to start an AWS EC2 instance.
Here is my main.tf:
resource "aws_instance" "ec2_example" {
ami = "ami-0767046d1677be5a0"
instance_type = "t2.micro"
tags = {
Name = "Test - Terraform EC2"
}
}Apply the above Terraform configuration by running the following commandterraform applys:
After successful execution of the above Terraform code, let’s verify the debug logs available at /home/vagrant/terraform-ec2-aws/terraform-debug.log.
Here is the screenshot taken from the terraform-debug.log file which shows the DEBUG logs along with the INFO logs:
Here are some sample logs from the terraform-debug.log:
2021-12-15T20:19:28.645Z [INFO] Terraform version: 1.1.0
2021-12-15T20:19:28.645Z [INFO] Go runtime version: go1.17.2
2021-12-15T20:19:28.645Z [INFO] CLI args: []string{"terraform", "init"}
2021-12-15T20:19:28.645Z [DEBUG] Attempting to open CLI config file: /home/vagrant/.terraformrc
2021-12-15T20:19:28.645Z [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021-12-15T20:19:28.645Z [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-12-15T20:19:28.645Z [DEBUG] ignoring non-existing provider search directory /home/vagrant/.terraform.d/plugins
2021-12-15T20:19:28.645Z [DEBUG] ignoring non-existing provider search directory /home/vagrant/.local/share/terraform/plugins
2021-12-15T20:19:28.645Z [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/pluginsNow that we have seen in the previous section on how to set up the TF_LOG and TF_LOG_PATH, let’s take a look at an example of how to put some custom validation on the Terraform variable, and to see if the validation fails. For this, we are going to log an error_message.
Validation condition: Provision of the EC2 instance should be restricted to t2.nano, t2.micro, t2.small. If the developer wants to provision the t2.medium or any other higher instance, we should restrict the developer with the proper error messages.
Here is a Terraform script with a validation and error_message:
resource "aws_instance" "ec2_example" {
ami = "ami-0767046d1677be5a0"
instance_type = var.instance_type
tags = {
Name = "Terraform EC2"
}
}
variable "instance_type" {
description = "Instance type t2.micro"
type = string
default = "t2.medium"
validation {
condition = can(regex("^[Tt][2-3].(nano|micro|small)", var.instance_type))
error_message = "Invalid Instance Type name. You can only choose - t2.nano,t2.micro,t2.small"
}
}If you look carefully into the variable resource:
- condition: We have defined the regex to allow only – t2.nano, t2.micro, t2.small
- error_message: We have set up an error message stating “Invalid Instance Type name. You can only choose – t2.nano,t2.micro,t2.small”
So, when the user tries to supply t2.medium then they should get the following error message:
Inspecting the Terraform state is an essential technique for troubleshooting Terraform configurations and understanding how your infrastructure is currently set up.
The Terraform state file holds a record of all the settings and attributes of resources defined, serving as a source of truth for Terraform to determine resource configurations and detect any drift from the desired state.
Use commands like terraform show to examine the current state and terraform plan to find differences between the configurations and the state during debugging. You can find problems like misconfigurations, inconsistencies, or state lock issues by looking in the state file itself – this will help you identify the core cause of any unusual behavior or failures in your infrastructure provisioning. If you are using remote state, and you want to inspect the state file, you can also use the terraform state pull command to pull the remote state locally and inspect it there.
By using different levels of debugging, Terraform generates a lot of data that can be hard to follow. Using external logging tools can greatly help with parsing, analyzing, and visualizing this information in a more digestible format. By forwarding Terraform logs to systems such as the ELK Stack (Elasticsearch, Logstash, Kibana), Splunk, or Grafana, you can take advantage of advanced search capabilities, real-time monitoring, and trend analysis.
These tools provide powerful filtering and query options, allowing you to quickly isolate issues, understand resource dependencies, and assess the impact of changes in your infrastructure.
This post is all about making yourself aware of the debugging and logging features provided by Terraform for troubleshooting the issues which you might face in your development, testing, staging, or production instances.
Setting up the correct info, debug and error message can significantly lessen your troubleshooting time as well as make your monitoring task. You can even dump the Terraform logs into log analyzer tools such as Splunk or ELK to get some visual graphs.
We encourage you also to explore how Spacelift makes it easy to work with Terraform. If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this. It supports Git workflows, policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box. You can check it for free by creating a trial account.
Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.
Terraform Management Made Easy
Spacelift effectively manages Terraform state, more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization and includes many more features.
