In this blog post, we will delve into the world of serverless computing with AWS Lambda and API Gateway and demonstrate how to leverage the capabilities of Terraform to streamline the deployment process. By combining the serverless aspect of AWS with the infrastructure-as-code capabilities of Terraform, we can achieve seamless, scalable, and easily maintainable serverless applications.
We will implement all the components required to host a serverless API on AWS serverless infrastructure, using API Gateway and Lambda functions. Further, we will also map the API Gateway endpoint with a custom domain using Route 53, implement authentication using Cognito, and secure the end-to-end communication using TLS certificates issued from ACM.
We will cover:
- Introduction to AWS Services and prerequisites
- Step 1: Create API Gateway
- Step 2: Create Lambda application code using NodeJS
- Step 3: Create an AWS Lambda function using Terraform
- Step 4: Integrate Lambda function with API Gateway
- Step 5: Provision appropriate access for the Lambda function
- Step 6: Enable the CORS
- Step 7: Test the deployment using Postman
- Step 8: Provision AWS Cognito user pool
- Step 9: Use Cognito Authorizer in API Gateway for authentication
- Step 10: Generate an authentication token from Cognito and access the API
- Step 11: Set up the domain name for the API
The complete source code used for the example discussed in this blog post is available here.
To follow along with the hands-on example discussed in this blog post, preliminary knowledge following AWS Services is required. Below is the introduction of these services, in short, explaining the purpose behind them. This is an important primer if you are new to these services while building the rest of the serverless application API.
What is a serverless architecture?
Serverless architecture has emerged as a transformative paradigm, enabling developers to focus solely on their code without the complexities of infrastructure management. AWS Lambda, coupled with API Gateway, offers a powerful solution for deploying applications in a serverless manner, dynamically scaling resources in response to demand.
However, managing the deployment of these resources efficiently requires a strategic approach.
What is AWS Lambda?
AWS Lambda stands as a serverless computing service that frees developers from the intricacies of server management. By allowing code to be executed in response to events, Lambda ensures optimal resource utilization and eliminates the need for provisioning and scaling servers.
What is API Gateway?
API Gateway offers a managed solution for creating and deploying APIs seamlessly. It acts as a gateway for RESTful APIs, WebSocket-based communication, and HTTP endpoints, providing features like authentication, throttling, and caching.
This service simplifies the process of exposing backend services to the web as well as privately while offering essential management and monitoring capabilities.
What is Amazon Cognito?
Amazon Cognito addresses the complexities of user identity management. As a comprehensive identity provider, it facilitates easy integration of authentication and authorization features into applications.
Cognito supports various identity sources, enabling developers to manage user registration, sign-in, and access control, thus enhancing the security and user experience of applications.
What is AWS Certificate Manager?
AWS Certificate Manager plays a pivotal role in the security of applications by simplifying the management of SSL/TLS certificates.
With automatic certificate renewal and provisioning, this service ensures that encrypted communication between clients and servers remains uninterrupted. This is crucial for maintaining the confidentiality and integrity of data exchanged over the internet.
What is Route 53?
Route 53 serves as a scalable domain name system (DNS) service. It translates user-friendly domain names into IP addresses, enabling users to access AWS resources reliably and seamlessly.
Route 53 offers domain registration, routing, and health-checking capabilities, making it an essential component for maintaining high availability and optimal performance of applications and services.
Our basic serverless API – which will be built in this blog post – is represented in the diagram below.
The end result will enable the users to fetch data served from the Lambda function by sending a POST request to the connected API exposed by API Gateway and mapped by the Route 53 domain.
The user will first have to authenticate against the Cognito authorizer to generate the authentication tokens, without which accessing the Lambda function will not be allowed by API Gateway.
To begin with, we first provision an API Gateway.
Although the API Gateway is dependent on the Lambda function, we can associate the same later when the Lambda function is read. For the sake of simplicity, we will create all the resources in separate Terraform files.
Create a file named “apigateway.tf”, and add the code below.
resource "aws_api_gateway_rest_api" "my_api" {
name = "my-api"
description = "My API Gateway"
endpoint_configuration {
types = ["REGIONAL"]
}
}
resource "aws_api_gateway_resource" "root" {
rest_api_id = aws_api_gateway_rest_api.my_api.id
parent_id = aws_api_gateway_rest_api.my_api.root_resource_id
path_part = "mypath"
}Here, we are creating an API Gateway API and adding a root resource to the same. When this configuration is provisioned, the resource will be accessible at “<URL>/mypath”. “mypath” is the path name. We can create multiple resources with different paths.
As seen in the reference screenshot below, every API Gateway resource consists of four components: Method request, Integration Request, Integration Response, and Method Response. These components are responsible for processing incoming requests and outgoing responses in various ways.
We will see how we use the Method Request block to implement authorization in upcoming steps. For more information, refer to this document.
We write the Terraform configuration below to add a POST method on our API Gateway resource.
The code below has four blocks corresponding to each block shown in the above screenshot.
resource "aws_api_gateway_method" "proxy" {
rest_api_id = aws_api_gateway_rest_api.my_api.id
resource_id = aws_api_gateway_resource.root.id
http_method = "POST"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "lambda_integration" {
rest_api_id = aws_api_gateway_rest_api.my_api.id
resource_id = aws_api_gateway_resource.root.id
http_method = aws_api_gateway_method.proxy.http_method
integration_http_method = "POST"
type = "MOCK"
}
resource "aws_api_gateway_method_response" "proxy" {
rest_api_id = aws_api_gateway_rest_api.my_api.id
resource_id = aws_api_gateway_resource.root.id
http_method = aws_api_gateway_method.proxy.http_method
status_code = "200"
}
resource "aws_api_gateway_integration_response" "proxy" {
rest_api_id = aws_api_gateway_rest_api.my_api.id
resource_id = aws_api_gateway_resource.root.id
http_method = aws_api_gateway_method.proxy.http_method
status_code = aws_api_gateway_method_response.proxy.status_code
depends_on = [
aws_api_gateway_method.proxy,
aws_api_gateway_integration.lambda_integration
]
}Points to note:
- We have set the Method and Integration method as “POST”, as intended.
- We have set authorization as “NONE”. This will be changed in the upcoming steps.
aws_api_gateway_integrationresource block is configured with type “MOCK”. This will be changed to “AWS” in later steps.
Similarly, repeat the above four resource blocks for the “OPTIONS” method as well. It is not required to create OPTIONS methods, but it does help in CORS preflight browser requests. We will cover CORS in later steps.
Note: Complete code with OPTIONS request is available in this file.
Every time the API Gateway configuration changes, we have to explicitly deploy the same on a stage of our choice. Thus we add the configuration block below to let Terraform perform this step.
resource "aws_api_gateway_deployment" "deployment" {
depends_on = [
aws_api_gateway_integration.lambda_integration,
aws_api_gateway_integration.options_integration, # Add this line
]
rest_api_id = aws_api_gateway_rest_api.my_api.id
stage_name = "dev"
}Apply this Terraform configuration for API Gateway, and make sure the it is created in the AWS Console.
Before we provision a Lambda function, let us first create a simple NodeJS application that simply returns a string, as seen below.
Create a file named “index.js” and add the following code. This is a simple Lambda function handler that responds with a greeting for any incoming request.
exports.handler = async (event) => {
// Your Lambda function logic here
const response = {
statusCode: 200,
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({
message: "Hello from Lambda!"
// Add more data as needed
})
};
return response;
};This is part of the application code. This file could potentially be part of a separate repository where all the Application code is developed separately.
That repository can then be used by this Terraform configuration to provision infrastructure resources.
Before we create the configuration for the Lambda function itself, we need to zip the source code file – index.js. Thankfully, it is also possible to generate this zip file using the “archive_file” data resource of Terraform.
Create the data source as below.
data "archive_file" "lambda_package" {
type = "zip"
source_file = "index.js"
output_path = "index.zip"
}The configuration above is quite straightforward, it takes the index.js as the source file and outputs the index.zip file. We then proceed to create the configuration for the Lambda function and refer to this zip file as shown below.
The role attribute is mandatory, which accepts the IAM role with appropriate permissions, and the same is created in aws_iam_role resource block.
resource "aws_lambda_function" "html_lambda" {
filename = "index.zip"
function_name = "myLambdaFunction"
role = aws_iam_role.lambda_role.arn
handler = "index.handler"
runtime = "nodejs14.x"
source_code_hash = data.archive_file.lambda_package.output_base64sha256
}
resource "aws_iam_role" "lambda_role" {
name = "lambda-role"
assume_role_policy = jsonencode({
Version = "2012-10-17",
Statement = [
{
Action = "sts:AssumeRole",
Effect = "Allow",
Principal = {
Service = "lambda.amazonaws.com"
}
}
]
})
}Apply this updated Terraform configuration and verify if the Lambda function with our NodeJS application code is created.
Read also – How to create an AWS IAM role with Terraform.
In Step 1 of this blog post, we created an API Gateway using Terraform and set its integration type to be “Mock”. And in Step 3, we created the Lambda function using Terraform independently.
In this step, let’s integrate the Lambda function with the API Gateway so that the incoming requests on the API Gateway are passed to the Lambda function.
To do this, in the aws_api_gateway_integration resource for root resource (/mypath) replace the type from MOCK to AWS. This change informs AWS that this API Gateway would be triggered by one of the other AWS services being provisioned.
To specify the service responsible for triggering this Lambda function, provide its ARN in the “uri” attribute in the same resource.
The updated Terraform configuration looks like this.
resource "aws_api_gateway_integration" "lambda_integration" {
rest_api_id = aws_api_gateway_rest_api.my_api.id
resource_id = aws_api_gateway_resource.root.id
http_method = aws_api_gateway_method.proxy.http_method
integration_http_method = "POST"
type = "AWS"
uri = aws_lambda_function.html_lambda.invoke_arn
}Apply this Terraform configuration and make sure that the trigger of the Lambda function is associated with API Gateway, as seen in the screenshot below.
However, if we try to test the API Gateway resource from the interface above, it throws an error, as seen below
Somewhere in the logs, we can find the reason for this error.
Execution failed due to configuration error: Invalid permissions on Lambda functionThe error message above is quite clear. We have not provisioned appropriate policy and role to our Lambda function.
To do this, add the configuration below.
resource "aws_iam_role_policy_attachment" "lambda_basic" {
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
role = aws_iam_role.lambda_role.name
}
resource "aws_lambda_permission" "apigw_lambda" {
statement_id = "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.html_lambda.function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_api_gateway_rest_api.my_api.execution_arn}/*/*/*"
}In the first block, we attach the AWSLambdaBasicExecutionRole to the IAM role we created in Step 3.
Next, we allocate permissions to the Lambda function to “AllowExecutionFromAPIGateway”.
Apply this updated configuration again, and test the API.
This response should confirm that we have successfully integrated API Gateway to the Lambda function.
API Gateway throws CORS error if not explicitly set. To avoid the same, in this step, we enable CORS headers in the method response and integration response of our API Gateway.
For our root API Gateway (/mypath) resource, update the method and integration response blocks with nested response_parameters block, as shown below.
resource "aws_api_gateway_method_response" "proxy" {
rest_api_id = aws_api_gateway_rest_api.my_api.id
resource_id = aws_api_gateway_resource.root.id
http_method = aws_api_gateway_method.proxy.http_method
status_code = "200"
//cors section
response_parameters = {
"method.response.header.Access-Control-Allow-Headers" = true,
"method.response.header.Access-Control-Allow-Methods" = true,
"method.response.header.Access-Control-Allow-Origin" = true
}
}
resource "aws_api_gateway_integration_response" "proxy" {
rest_api_id = aws_api_gateway_rest_api.my_api.id
resource_id = aws_api_gateway_resource.root.id
http_method = aws_api_gateway_method.proxy.http_method
status_code = aws_api_gateway_method_response.proxy.status_code
//cors
response_parameters = {
"method.response.header.Access-Control-Allow-Headers" = "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'",
"method.response.header.Access-Control-Allow-Methods" = "'GET,OPTIONS,POST,PUT'",
"method.response.header.Access-Control-Allow-Origin" = "'*'"
}
depends_on = [
aws_api_gateway_method.proxy,
aws_api_gateway_integration.lambda_integration
]
}Apply this configuration and make sure everything works as expected.
Additionally, verify if the header settings are updated in the integration response as below:
And in method response as this:
Before we proceed to implement the authentication and custom domain mapping, let’s create a Postman test. This will be useful in upcoming steps when we work with authentication tokens.
Currently, our API is public. The POST method we provisioned is accessible to everyone without any credentials.
Navigate to API Gateway > API > Stages > dev > POST, and copy the URL from the top, as seen in the screenshot below.
Create a Postman test using the above URL, and set the method to POST.
Send the request to API Gateway by clicking on the Send button. The response should be a success with a 200 status code and the body containing the message returned from the Lambda function.
With this response, we can confirm that the API Gateway and Lambda functions are accessible publicly.
Note that if it throws an error, wait for a while and try again after a few minutes.
To implement Cognito authentication on our API Gateway, first begin by creating the Cognito user pool.
The simple resource block below creates a Cognito user pool with a provided name.
resource "aws_cognito_user_pool" "pool" {
name = "mypool"
}To integrate this user pool with our API Gateway (or our application), we need to create a user pool client. We do the same by adding the aws_cognito_user_pool_client resource block.
This resource accepts several parameters that enable various ways for the clients to authenticate themselves against the Cognito user pool. We have also associated this client with the Cognito user pool using the user_pool_id attribute.
resource "aws_cognito_user_pool_client" "client" {
name = "client"
allowed_oauth_flows_user_pool_client = true
generate_secret = false
allowed_oauth_scopes = ["aws.cognito.signin.user.admin","email", "openid", "profile"]
allowed_oauth_flows = ["implicit", "code"]
explicit_auth_flows = ["ADMIN_NO_SRP_AUTH", "USER_PASSWORD_AUTH"]
supported_identity_providers = ["COGNITO"]
user_pool_id = aws_cognito_user_pool.pool.id
callback_urls = ["https://example.com"]
logout_urls = ["https://sumeet.life"]
}Finally, since we are using Terraform to provision everything, let us also provision a test user to test our API Access.
resource "aws_cognito_user" "example" {
user_pool_id = aws_cognito_user_pool.pool.id
username = "sumeet.n"
password = "Test@123"
}Apply this configuration and make sure the Cognito user pool, user pool client, and user is created, as seen in the screenshots below.
User pool with user:
User pool client:
Note the user pool ID and client ID, as this will be used in generating the access tokens for the user.
Now that we have provisioned the Cognito user pool, user pool client, and the user as well, let us integrate it with our API Gateway to restrict public access. In AWS, we can add Cognito Authorizer to API Gateway.
When the authorizer is enabled, any incoming request token is first validated against this Cognito user pool before Lambda is triggered.
To do this, add the aws_api_gateway_authorizer resource block as shown below.
resource "aws_api_gateway_authorizer" "demo" {
name = "my_apig_authorizer2"
rest_api_id = aws_api_gateway_rest_api.my_api.id
type = "COGNITO_USER_POOLS"
provider_arns = [aws_cognito_user_pool.pool.arn]
}Here we have given a name to our authorizer, associated it with the appropriate REST API, set the type as “COGNITO_USER_POOL”, and provided the ARN of the corresponding user pool.
Note that when we provisioned API Gateway in step 1, we set the authorization attribute to “NONE”. This is where we change it to “COGNITO_USER_POOL” as well. We also mention the authorizer ID generated from the resource block above.
The updated configuration for the API Gateway method looks like this:
resource "aws_api_gateway_method" "proxy" {
rest_api_id = aws_api_gateway_rest_api.my_api.id
resource_id = aws_api_gateway_resource.root.id
http_method = "POST"
//authorization = "NONE" // comment this out in cognito section
authorization = "COGNITO_USER_POOLS"
authorizer_id = aws_api_gateway_authorizer.demo.id
}If we navigate to API Gateway Method Request, we should be able to see that the auth is set to the name of the authorizer we provided above.
To make sure that the authorizer has taken effect, go back to the Postman request, and click on send again.
It should correctly return the 401 status code like so:
We have successfully protected our API from public access.
Since the API is now protected, it now needs an access token to invoke the Lambda function and fetch the data.
To generate the access token, for this example, we use AWS cognito-idp CLI.
There are multiple ways to generate the tokens, and it depends on which auth flows we have enabled in the user pool client. In step 8, we also noted the user pool ID and client ID – we will need this information to generate the tokens now.
Run the CLI command below and observe the output. Replace the IDs with your Cognito pool IDs.
aws cognito-idp admin-initiate-auth --user-pool-id <USER_POOL_ID> --client-id <CLIENT_ID> --auth-flow ADMIN_NO_SRP_AUTH --auth-parameters USERNAME=sumeet.n,PASSWORD=Test@123Here, we are basically authenticating our Test user against the Cognito User pool we created before and, in return, getting the access token back.
This command should return the Access Token, Token Type, Expires In time, Refresh token, and Id token, as seen in the example below.
{
"ChallengeParameters": {},
"AuthenticationResult": {
"AccessToken": "ACCESS_TOKEN",
"ExpiresIn": 3600,
"TokenType": "Bearer",
"RefreshToken": "REFRESH_TOKEN",
"IdToken": "ID_TOKEN"
}
}To test the validity of these tokens, copy the value in IdToken, and navigate to the API Gateway > authorizers, as seen below.
Click on “Test”.
In the window that appears, paste the IdToken in the Authorization (header) field, then click on the Test button. The response indicates that the Cognito authorizer has successfully identified the token for the existing user.
Let us try to access the API from Postman again using this IdToken. Before clicking on the Send button, create a new header and name it “Authorization”.
In the value, paste the IdToken in this format – “Bearer <IdToken>”.
As seen in the screenshot below, we are able to access the Lambda function’s response again. Thus we are able to access the API Gateway again only with correct credentials.
Often it is desirable to access these backend APIs via a meaningful domain name – usually a subdomain like “api.example.com”.
In this step, we will map a subdomain to access this API. As a prerequisite, make sure that you have purchased a domain name from Route 53, and the DNS are managed in Route 53 hosted zone.
For this example, I have already bought a domain “sumeet.life” and the DNS records are also managed in the same AWS account. We are going to map our API Gateway endpoint to api.sumeet.life so that we can send POST requests to “https://api.sumeet.life/mypath”, instead of “https://6ihp8jetll.execute-api.eu-central-1.amazonaws.com/dev/mypath”.
Since the domain is already bought, we would use the aws_route53_zone data source to access the same, as shown below.
data "aws_route53_zone" "my_domain" {
name = "sumeet.life"
private_zone = false
}Next, we create a CNAME record for “api.sumeet.life” to map it to the API Gateway Endpoint, as shown below. Notice how we are dynamically picking up the initial part of the API Gateway endpoint URL.
resource "aws_route53_record" "custom_domain_record" {
name = "api" # The subdomain (api.sumeet.life)
type = "CNAME"
ttl = "300" # TTL in seconds
records = ["${aws_api_gateway_rest_api.my_api.id}.execute-api.eu-central-1.amazonaws.com"]
zone_id = data.aws_route53_zone.my_domain.zone_id
}Apply this configuration and make sure the appropriate DNS CNAME record is created in the Route 53 hosted zone, as seen below.
This is not the end. We now have to register this custom domain with the API Gateway domain. If we try to send the Postman request now, it will respond with 403 Forbidden.
To create an API Gateway custom domain, first provision an ACM certificate as shown below.
resource "aws_acm_certificate" "my_api_cert" {
domain_name = "api.sumeet.life"
provider = aws.aws_useast1 # needs to be in US East 1 region
subject_alternative_names = ["api.sumeet.life"] # Your custom domain
validation_method = "DNS"
}A couple of things to note:
- When this certificate is provisioned, we need to manually navigate to ACM and create corresponding Route 53 records by clicking on the button “Create records in Route 53” as seen in the screenshot below.
- The certificates should be created in the US-East-1 region. The API Gateway domains throw errors otherwise. That is why we are using a provider alias for US-East-1.
Make sure that once these records are created, the certificate status is issued and the domain state is “success”. It may take a few minutes for this to happen.
Next, we create the API Gateway domain name and map its base path with “dev” stage. This is done in the configuration blocks below.
If you get the same output, we have successfully accomplished what we were set to do at the beginning of this blog post. To test this, go back to the Postman request.
This time instead of using the API Gateway endpoint, replace the same with the (sub)domain you have mapped (api.sumeet.life) in this case, and click on Send.
Note that since we have included the “/dev” part in API mappings, remove the same from the URL path. The response should be a success if the Authorization token is valid.
If it throws an “unauthorized” error, try to regenerate the token using the aws cognito-idp command mentioned above.
In this post, we have covered a lot of ground. We have briefly touched upon topics like authentication, domains, API Gateway, etc. which essentially are quite deep. However, the intention of the steps described here is to help you understand and get going with the first end-to-end serverless application API.
We encourage you also to explore how Spacelift makes it easy to work with Terraform. If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this. It supports Git workflows, policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box.
Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.
The Most Flexible CI/CD Automation Tool
Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.
