How to Manage AWS Lambda Functions with Terraform: Tutorial

Serverless applications are becoming popular among the DevOps community because they do not require application server infrastructure. AWS Lambda is an event-driven offering from AWS that can help you run any application without the need for application servers.

In this blog post, we will discuss AWS Lambda and how to use Terraform to manage it. In the tutorial, we will start with the basics of the AWS Lambda function and then deploy a very simple Python app using it.

What we will cover:

• • What is the AWS Lambda function in Terraform?
  • What is AWS Lambda used for?
  • Terraform Lambda resources
    • Prerequisites
    • Step 1: Set up the IAM Roles and Policies
    • Step 2: Write the Python application
    • Step 3: Create a ZIP file of the Python application
    • Step 4: Add the aws_lambda_function
    • Step 5: Run and execute Terraform main.tf
    • Step 6: Verify the Lambda function from the AWS console
  • Example: Terraform configuration with AWS Lambda function
  • AWS Lambda Terraform module

AWS Lambda is a serverless compute service that executes code in response to events without the need for provisioning or managing servers. Although you can create and run Lambda functions from the AWS web console, Terraform offers a great solution for managing Lambda functions through Infrastructure as Code (IaC). 

In Terraform, an AWS Lambda function is defined using the aws_lambda_function resource. This resource specifies key details such as the function’s name, runtime environment (e.g., Python, Node.js), handler method, and source code location (either inline or from an S3 bucket). Additionally, configurations for memory, timeout, environment variables, and permissions can also be set up within this resource.

For example, deploying a Python script to AWS Lambda with Terraform requires the following steps:

1. Install Terraform
2. Configure the AWS account
3. Set up the IAM roles and policies
4. Write the Python application
5. Create a ZIP file of the Python application
6. Add the aws_lambda_function
7. Run and execute Terraform main.tf
8. Verify the Lambda function from the AWS console

AWS Lambda function is very flexible and supports various use cases:

1. Event-driven applications — AWS Lambda can be triggered from a number of different event sources. Examples include triggering when a database row is updated, and triggering when the message count in a queue is greater than a certain number.
2. Web and mobile backends — For a Mobile touchscreen, GUI event can be integrated with AWS Lambda using AWS API Gateway to trigger the Lambda function by calling the microservice-http-endpoint. (Learn how to create API Gateway using Terraform)
3. Machine learning and data processing — As AWS Lambda supports Python, it is well suited for machine learning and data analytics applications. ML and Data Analytics applications rely heavily on Python and its libraries (Numpy, TensorFlow, MatPlotlib, Scipy, etc.).

Let’s take a deeper look at Terraform Lambda resources and how they provision Lambda functions and other resources.

• aws_lambda_function — The aws_lambda_function resource creates an AWS lambda function inside your AWS account. You can provide the code via an S3 bucket or local ZIP file and set configuration details such as the handler, the runtime, and other parameters as well.

• aws_lambda_alias — By using the aws_lambda_alias resource in Terraform, you can set up an alias for your AWS lambda function. With this alias, you can point to a specific version of your lambda function, making it easier to use different versions of your functions.

• aws_lambda_permission — Lambda permissions give other services permissions to invoke your lambda functions.

• aws_lambda_layer_version — A layer version resource creates a lambda layer for your functions which is a zip archive containing shared code or libraries shared by your lambda functions.

• aws_api_gateway_rest_api — The aws_api_gateway_rest_api creates an API Gateway in AWS. You can define endpoints that invoke Lambda functions, acting as a front door for your applications.

• aws_cloudwatch_log_group — A cloud watch log group can be used to monitor, store, and access log files from various AWS services.

Let’s start with the prerequisites for this managing AWS Lambda function with Terraform tutorial.

1. You should have Terraform installed onto your machine (see our tutorial here: How to Download and Install Terraform on Windows, Linux & macOS)
2. You must have an AWS account. 

Before proceeding further, make sure Terraform is installed by running the command – $ terraform -version

Now, here’s how to manage AWS Lambda functions:

The first step is to set up an IAM Role for your Lambda function and any policies that it requires. The exact policies required will depend on what your function does and what services it needs to access. Since this article is mainly focused on explaining how to manage Lambda functions via Terraform, we will keep the IAM Roles and Policies very basic.

Create main.tf and add IAM role

Let’s start by creating a main.tf file and inside the main.tf file, creating a role called Spacelift_Test_Lambda_Function_Role.

Add an IAM policy

After creating the IAM Role, let’s create an IAM Policy to manage the permissions associated with the role. As this is a basic application, we will be assigning the following permissions: 

1. logs:CreateLogGroup
2. logs:CreateLogStream
3. logs:PutLogEvents

Add the following IAM Policy resource block to main.tf:

Attach IAM policy to IAM role

Now that we have created an IAM policy and IAM role for the Terraform managed AWS Lambda function, let’s attach both the IAM policy and IAM role to each other:

Read more about Creating IAM Policies with Terraform.

Since the IAM Role and IAM policy have now been created, let’s call on the Python application that we will be running on AWS Lambda.

1. Create a directory python parallel to main.tf.
2. Inside the directory python, create a file index.py.
3. Add the following python function to it:

As in the previous step, we created a Python file index.py. Now, we need to create a ZIP file because aws_lambda_function needs the code to be stored in a ZIP file before uploading it to AWS.

This ZIP file we are going to upload and submit this to the AWS Lambda function:

Alright. Now we have everything (IAM role, IAM policy, Python code) in place. Let’s write down the aws_lambda_function resource block:

Here are a few things you should keep in mind while writing aws_lambda_function resource block:

• Runtime — You should mention the correction runtime that AWS Lambda will use to run your Lambda function. Currently, AWS Lambda supports Node.js, Python, Java, Ruby, Go, and NET.
• IAM role — Always mention the correct IAM role that you have created for the Lambda function.
• Depends_on — Mention the correct IAM policy attachment block where you have attached the IAM role to the IAM policy. This is a sanity check to make sure that the IAM Policy and IAM roles are in place before the lambda function is created.

Here is the complete main.tf file:

Let’s run the following commands in sequence:

1. $ terraform init

2. $ terraform plan

Here is the remaining output of the terraform plan:

3. $ terraform apply

Here is the remaining output of the terraform apply:

Let’s head over to the AWS Console and do the test run. From the AWS dashboard, you can start by searching Lambda from the search bar:

Click on Lambda to see the Lambda function we have provisioned using Terraform:

Click on it, then use the Test button to run the Lambda function:

A new window will open where you need to specify the event name. As you can see in the picture below, we have assigned the event name MyCustomMessage and entered a custom message as key1.

Now, click on the Test tab and then on the orange Test Button:

As you can see, it has returned the message Hello Spacelift, which we have written inside the python file.

Here’s another example of how to define a simple AWS Lambda function in Terraform.

For this example, we will use a hello world Lambda function:

I’ve saved this into a file called hello.py.

The first step we will do in our Terraform code is to define our provider and a role for the Lambda function:

The above role uses an AWSLambdaBasicExecutionRole policy that provides the necessary permissions to execute lambda functions and write logs to AmazonCloudwatch.

Next, we will define an archive_file datasource that creates the lambda archive for us:

In the end, we define the lambda function to use the role and the archive we’ve created above and specify the handler of our function in this format: function_name.lambda_handler. In addition, we are creating an aws_cloudwatch_log_group to get the logs of our Lambda execution.

Now, let’s apply the Terraform code:

We are now ready to invoke the lambda function, so we will do that and save the output in a file called output.txt.

Let’s check the output:

We can also check the log event in CloudWatch:

There is an AWS Lambda Terraform module available in the Terraform registry. Let’s use this module to build something similar to what we’ve done in the previous example.

This will create almost the same resources as in our above case. The only difference is that the permissions are slightly different, and the archive is created using a null resource with a local-exec provisioner that runs a Python script.

Let’s run the code and see the output:

Now we can invoke the function to ensure that everything is working properly:

For this part, we will create a GitHub repository based on the code we initially built. The example GitHub repository is here.

Now, let’s go to our Spacelift account and create a stack:

Add a name to your stack, select a Space, optionally add Labels and a Description, and then click on continue.

In the Connect to source code, select your VCS provider and the repository containing the code.

In the Choose vendor step you can accept the defaults: use the latest FOSS version of Terraform.

At this point, the stack is created, but we should also consider how to authenticate to AWS. For that, I will attach a cloud integration. If you don’t know how to create a cloud integration for AWS, check out this guide.

Add your integration, select both read and write and click on Attach. Then you can click on go to summary and confirm.

Let’s trigger a run on our stack to create the resources:

After the plan finishes, we can check what will be created in a human-readable format:

Let’s confirm the run and wait for the resources to be created:

Now, we can invoke the function using a task:

As you can see, the function was executed successfully and the message has been printed.

This article is meant to familiarize you with using Terraform to manage AWS Lambda functions. Although the article uses Python, you can satisfy your business needs using other programming languages such as GO, Java, Ruby, or .NET. The core concepts of setting AWS Lambda with Terraform remain the same regardless of your language choice.

If you want to elevate your Terraform management, create a free account for Spacelift today or book a demo with one of our engineers.

Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.