Managing AWS Security Groups Through Terraform

Virtual Private Cloud (VPC) is one of the fundamental building blocks of AWS. VPCs are private virtual networks that resemble traditional networks and provide secure, isolated environments to host and protect AWS resources like EC2 instances.

Security for the resources within the VPC can be configured using security groups and a network access control list (NACL).

Let’s understand how we can use them to protect resources within VPC and improve the overall security posture.

Read more about how to build AWS VPC using Terraform and how to create AWS EC2 instance using Terraform.

What are security groups?

Security groups are one of the most important pillars that support VPC security. They are software-defined network firewalls defined inside a VPC that allow or deny traffic to resources based on the inbound and outbound rules.

They support stateful Layer 3 (Network layer) and Layer 4 (Transport layer) filtering capabilities.

At the resource level, security groups work to secure the flow of traffic allowed to leave and reach them. They serve as the final line of defense.

They are most commonly associated with EC2 instances, but their applications are not limited to them. They are widely used with a variety of other services such as CodeBuild, EKS, and so on.

The figure below shows how security groups secure traffic flow to and from resources.

What makes up a security group?

A security group is composed of rules. A rule consists of the following components:

• Protocol
• Port range
• Source
• Destination

A security group has separate rule sets for incoming and outgoing traffic.

An inbound rule consists of a protocol, a port(s), and a source.

For example, the below rule allows all traffic to reach resources associated with the security group.

An outbound rule consists of a protocol, a port(s), and a destination.

For example, the below rule allows all traffic to leave resources associated with the security group.

Let us inspect an existing security group to understand the rules better.

Exploring the default security group

Let us inspect the default security group to understand what the rules look like.

1. Open the EC2 dashboard.

2. Click on “Security Groups” in the left nav under “Networks and Security”.
3. Explore the inbound and outbound rules for the default security group and other details.

Key observations

1. The default security group is associated with a VPC. The VPC ID can be found under the “VPC ID” tab.
2. By default, the security group has one inbound and one outbound rule.
3. All traffic can reach the associated resources thanks to the inbound rule.
4. The outbound rule allows all traffic destined for everywhere (denoted by 0.0.0.0) to leave the associated resources.

We can always create new security groups with whatever rules we need. In the following section, we will learn how to do so.

Note: Please keep in mind that we can only create 2500 security groups per region.

Security consideration: Keep the number of security groups to a minimum to avoid making mistakes when managing too many security groups.

1. Navigate to the “Security groups” panel.
2. Click the “Create security group” button to create a new security group.
3. Give the security group a name as web-server-sg and a description as Allow SSH to the web server.

Note: The name of the security group has to be unique within the VPC and cannot exceed 255 characters. Another thing to keep in mind is that the security group’s name cannot start with sg-.

4. For the time being, do not add any inbound rules and proceed with the suggested outbound rule, which allows all traffic to flow everywhere (0.0.0.0).
5. Finally, click on the “Create security group” button.

Congratulations! You have created your first security group 🎉

Security groups are only effective when attached to an AWS resource. Next, we will attach it to an EC2 instance to see it in action.

Attaching Security Groups to AWS Resources

In this tutorial, we will create an EC2 instance and attach the newly created security group to it. We will then try to connect to this instance via SSH.

Attaching a security group to an EC2 instance

1. Navigate to the EC2 dashboard and click on the “Launch instances” button.
2. Assign the instance a name as web-server.
3. Choose any Amazon Machine Image (AMI) and architecture.
   

4. Select the instance type as t2.nano.
5. Create a new key and name it web-server-key. The key will be downloaded automatically. We will use it later to SSH into our instance.
6. Make sure to select the default VPC and a public subnet under “Network settings”
7. Enable the “Auto-assign public IP” option.

8. Under the “Firewall (security groups)” tab, choose the option “Select existing security group” and select the security group we created earlier named web-server-sg.
   

9. Leave all other options unchanged and launch the instance.

With the security group now attached to our EC2 instance, let us try to connect to it via SSH.

Connecting to the EC2 instance via SSH

Open the details of the newly created EC2 instance and copy the public IP address.

2. Open a terminal and modify the permissions for the web-server-key.pem file we downloaded earlier to allow read and write permission for the owner using the below command.

3. Next, let us try to connect to the EC2 instance with our pem key using the below command.

After a while, the SSH request times out with the following error.

This is expected as our request is being blocked by the security group that we have attached to the EC2 resource. Remember that the security group we created does not yet have any inbound rules.

We can resolve this issue by configuring an inbound rule to allow incoming traffic on port 22 (SSH) from our IP address.

Adding rules to an existing security group

1. 1. Open the security groups panel and select the security group attached to our EC2 instance named web-server-sg.
   2. Under the “Inbound rules” tab, click the “Edit inbound rules” button, followed by the “Add rule” button.

3. Choose SSH as the protocol and My IP as the source (allowing SSH through your IP only). Optionally, add the description as Allow SSH from my network.

Security consideration: For protocols such as SSH and RDP, only specific IP addresses should be allowed. Specifying 0.0.0.0 opens access to everyone, which is a security risk.

It should be noted that the source and destination do not have to be IP addresses. Other security groups or prefixes can also be used.

4. Save the rule and attempt to reconnect to the EC2 instance via SSH.

Reconnecting to the EC2 instance via SSH

We’ll use the same SSH command as before to see if we can connect after adding the inbound rule to the security group.

Awesome! We successfully connected when we added an inbound rule for SSH.

You can also try switching to a different network to confirm that the security group is blocking the traffic.

In the following sections, we’ll go over some of the key characteristics of security groups, as well as how to manage security groups with Terraform.

When working with security groups, it is critical to be aware of a few key characteristics.

• There can only be “allow rules” in security groups, not “deny rules”
  AWS blocks all traffic to resources associated with security groups by default. The allow rules filter and allow traffic that is relevant.
• A resource can have multiple security groups attached to it
  When multiple security groups are attached to a single resource, all rules are aggregated to form a single rule set that determines whether or not the traffic is allowed.
• Security groups are stateful
  Being stateful means that with an outbound rule that allows traffic to flow out of the resource also allows the response to reach back regardless of whether an inbound rule is present or not. To better understand this analogy, please refer to this StackOverflow answer.

Handling infrastructure at scale manually is an extremely difficult task. Terraform and other IaC tools make it much easier for us to create and manage infrastructure at scale thanks to their declarative approach and state management.

Let’s look at how we can easily create and manage security groups with Terraform.

You will need to have Terraform installed on your machine and your AWS credentials configured in the shell to follow this tutorial.

Note: We will keep using the same main.tf file and make changes to it.

You can also take a look at how platforms like Spacelift can help you and your organization fully manage cloud resources within minutes. Spacelift is a CI/CD platform for infrastructure-as-code that supports tools like Terraform, Pulumi, Kubernetes, and more. For example, it enables policy-as-code, which lets you define policies and rules that govern your infrastructure automatically. You can even invite your security and compliance teams to collaborate on and approve certain workflows and policies for parts that require a more manual approach.

Creating a security group

We’ll set up a security group with an inbound rule that allows only HTTPS traffic and an outbound rule that allows all traffic.

1. Create a file with the name as main.tf and copy the below HCL code snippet to this file.

2. Run the terraform plan command in the terminal where the file resides. The plan should look something like the one below.

3. Run terraform apply to deploy the security group once the plan has been verified.

Great work! We have successfully deployed a new security group through Terraform 🎉

Adding rules to security groups

There are two ways to add rules to a security group in Terraform:

• Inline rules: Rules defined with the aws_security_group terraform resource

• Standalone rules: Rules defined separately using the aws_security_group_rule terraform resource

Note: Keep in mind that these are the two ways to add rules to a security group through Terraform, which at the end of the day, correspond to the same thing on AWS.

Terraform advises that inline rules should not be used in conjunction with standalone rules. Using both of them together can cause conflicts.

We’ll use standalone rules as we’ve already seen inline rules when creating a security group. Before adding any new rules, we will first convert the existing inline rules to standalone rules in the same file.

Converting inline security group rules to standalone rules

1. Set the ingress and egress attribute to an empty array.
   Note: Terraform doesn’t detect any changes if we completely remove ingress and egress attributes from aws_security_group. This issue can be tracked here. The workaround is to set the ingress and egress attributes to [].

2. Run the terraform plan command and then the terraform apply command to remove the existing rules before adding any standalone rules to avoid any conflicts.
3. Add standalone security group rules for HTTP ingress and all egress. Do not forget to remove the inline ingress and egress attributes from the aws_security_group resource to avoid any conflicts.

4. Run terraform plan again. The changes should reflect the two rules being added.

5. Finally run the terraform apply command to deploy the changes.

Awesome! We successfully added standalone rules to our security group.

Attaching a security group to an EC2 resource

In this tutorial, we will learn how to attach a security group to an EC2 instance.

1. Add Terraform resource to spin up an EC2 instance.
   

Note: We do need to create the web-server-key again as we had previously created it through the console.

2. Add key vpc_security_group_ids  with the value as [aws_security_group.web_server_sg_tf.id] to attach the previously created security group to this EC2 instance.

3. Run the terraform plan command and review the changes before running the terraform apply command.
4. Check the AWS console to verify if the security group was attached successfully.

Awesome! Our security group is attached to our EC2 instance as expected.

In this tutorial, we will learn how to refer to a security group created outside of terraform using a Terraform data resource.

We will first create a security group outside of Terraform and then add an inbound SSH rule to it using Terraform.

Referring to a single security group

1. Create a security group by the name security-group-managed-outside-terraform.

2. To refer to the security-group-managed-outside-terraform security group, use the aws_security_group data resource and provide the security group id in the main.tf file.

Relevant changes are shown below.

3. Add an inbound rule to this security group that allows SSH from the default VPC.
   

Note: We are using an id from the data resource, which is kind of redundant as we already knew the id and used it to fetch the security group. But in real-world scenarios, this can be useful to get other crucial information such as associated VPC id and more.

4. Run the terraform plan command. Notice that terraform finds the correct security group and plans to add an inbound rule to it.

5. Run the terraform apply command to deploy the changes.

Referring to multiple security groups

1. Let us create another security group called security-group-managed-outside-terraform-2 using the AWS console with the tag shown below.

2. Instead of referring to each of these security groups individually, we can use the aws_security_groups data resource and provide the tag value to refer to all of them at once.

Note: Previously, we used aws_security_group data source to refer to a single security group and now we are using aws_security_group(s) data source to refer to multiple security groups.

3. Modify the existing inbound SSH rule that we created earlier to add this rule to all the matching security groups. We will use the Terraform for_each meta tag to achieve this. Optionally, add the Terraform output to print all the matching security group ids.

4. As always, run the terraform plan command and review the changes. The plan should reflect the deletion of the previous rule and the creation of two new rules along with the output security_group_ids.

5. Finally, run the terraform apply command to deploy the changes to all relevant security groups.

Next, we will learn to import security groups existing outside terraform into terraform.

Importing existing security groups under terraform

When migrating an existing resource to terraform, it is not always feasible to delete and recreate it. In such cases, we can import resources directly into Terraform. Let us look at how we can easily bring existing security groups from outside of Terraform into Terraform.

We will import one of the resources we created earlier into Terraform in this tutorial. To learn more, see our Importing Existing Infrastructure into Terraform tutorial.

1. Declare the resource you want to import in the main.tf file. Add as many details to it as possible.

2. Run the below command with an id of the security group you want to import.

3. Run the terraform plan command to ensure that no changes are reflected.

This signifies that the resource was imported successfully with no conflicts 🎉

At this point, we’ve learned about how to create and manage security groups using both the console and Terraform. Let’s take a look at some popular applications and how they can help improve the security posture of a resource/application.

Security groups are a key part of the security of resources on AWS. There are multiple use cases where they are required, such as:

• Managing HTTPS(443) or HTTP(80) traffic to a web server.
• Configuring access to RDS instances. (Learn how to create an AWS RDS instance using Terraform.)
• Permitting traffic only from specific address ranges to the corporate network.
• Limiting SSH access to compute instances.
• Managing traffic to and from Lambda functions and more

Their usage is not limited to the above examples but extends to NAT Gateways, ECS, EKS, Code Build, and more.

It is already evident that security groups are crucial for network security. As mentioned earlier, security groups are one of two security features available to control traffic in your VPC. NACLs are the other type. To better configure network security, it is important to understand how security groups are different from NACLs and when to use what.

Before we get into how security groups differ from NACLs, let us quickly understand what NACLs are.

Network access control list (NACL)

NACLs, like security groups, are firewalls that allow you to control incoming and outgoing traffic. The main distinction between NACLs and security groups is that NACLs operate at the subnet level, whereas security groups operate at the resource level.

They can be used as a secondary control to define high-level guard rails for filtering traffic at the subnet level.

NACLs can help to improve a VPC’s security posture as they filter traffic regardless of whether a security group is associated with resources or not.

Security groups vs NACL

Although both security groups and NACLs are used to improve a VPC’s security posture, they have significant differences. The main points are as follows:

Security groups and NACLs can be used in tandem to implement the defense-in-depth strategy. NACLs supplement security groups and serve as an additional layer of protection.

This article would be incomplete without covering the best practices and the basics of compliance of security groups in AWS. Let us jump into it right away.

Network security is one of the most important aspects to take care of to improve the security posture. This becomes more difficult as the number of resources and accounts in an organization grows.

With many AWS accounts within an organization, how do we know if:

• Any of the security groups are over-permissive?
• Are there redundant or unused security groups?
• The security groups are compliant?
• Someone changes something with the existing security groups to make them less secure?

Furthermore, how do we make sure we are always compliant with the standard security benchmarks?

This is where AWS Firewall Manager and Security Hub come into the picture.

AWS Firewall Manager for Security Groups

AWS firewall manager solves all the above problems for us. It can track, audit, and even remediate issues with your security groups across all accounts.

It provides centralized visibility across all accounts to continuously monitor and manage violations and non-compliance.

The below picture depicts how the AWS Firewall manager automatically tracks non-compliance of the security groups.

AWS Firewall accomplishes this through the use of security group policies. These policies act as guard rails for creating new security groups or remediating issues with existing ones.

There are three types of security group policies:

• Common security groups
• Auditing and enforcement of security group rules
• Auditing and cleanup of unused and redundant security groups.

1. Common security groups policy help us to provision common or baseline security groups required in all accounts. For example, allowing SSH from only the corporate IP addresses.
2. Auditing of security groups assists us identity any security groups that do not adhere to the standard that we have created. These can be security groups with overly permissive or risky rules. It is also possible to automatically remediate non-compliant security groups.
3. An audit usage policy assists in detecting and removing unused security groups. It also helps in detecting and merging redundant security groups.

AWS Firewall Manager can send issues and alerts to AWS Security Hub for centralized compliance management.

AWS Security Hub for security groups

AWS Security Hub is a service that performs security checks. It aggregates all security alerts and issues in one place and enables us to visualize an organization’s security posture. It supports standards like:

• Center for Internet Security (CIS) AWS Foundations Benchmark
• AWS Foundational Security
• Payment Card Industry Data Security Standard (PCI DSS)

These standards include many security best practices for security groups, such as:

• CIS AWS Foundations 3.10: Ensure a log metric filter and alarm exist for security group changes
• PCI.EC2.5: Security groups should not allow ingress from 0.0.0.0/0 to port 22
• CIS AWS Foundations 4.2: Ensure no security groups allow ingress from 0.0.0.0/0 to port 3389
• PCI.EC2.2: VPC default security group should prohibit inbound and outbound traffic

and many more.

Any violations are automatically detected and aggregated. It enables us to configure automatic remediation to ensure that our security groups remain secure and compliant at all times.

Security groups are one of the most important security features for network security. They are simple yet powerful and thus find their uses across a wide variety of AWS services wherever networks are involved. When used correctly, they can significantly reduce attack vectors to network resources and improve the overall security posture.

Yet, it is crucial to remember that they are not a silver bullet to network security. It is just as important to have other defense mechanisms like NACLs, AWS Security Hub, AWS Firewall, etc, in place to have a more effective line of defense.

Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.