Solving the DevOps Infrastructure Dilemma: Enabling developer velocity with control đź’ˇ

Register for the webinar here →

General

What is Container Security? 10 Best Practices & Solutions

Container security best practices

In this article, we will delve into some best practices, tips, and tricks for securing your containers and Kubernetes (K8S) clusters with some configuration file examples.

We will cover:

  1. What is container security?
  2. Why is container security important?
  3. Container security architecture components
  4. Challenges and risks
  5. Best practices and suggested solutions

What is container security?

Container security refers to the practice of securing containerized applications and the infrastructure that supports them. Due to the ephemeral nature of containers and the dynamic deployments and scaling, container security measures should be automated and baked into every stage of the software development life cycle (SDLC).

Why is container security important?

As containerization plays a significant role in today’s software development, container security becomes more and more crucial for organizations.

Containers share the same OS kernel, so if a container is compromised, there’s a risk that an attacker could exploit potential vulnerabilities to affect the host system and other containers.

Because containers are created from images, and these images can contain vulnerabilities, an attacker exploiting these can gain access to other parts of the system. Containers often require secrets to operate, and improper management of these secrets can expose them to unauthorized access.

Ensuring security is paramount to protect sensitive data, maintain operational integrity, and safeguard against potential breaches that could have severe consequences.

Container security architecture components

Container security architecture is a comprehensive framework designed to protect containers throughout their lifecycle. This framework encompasses several key components, each focusing on different aspects of container security. Let’s look into the primary component:

  • Container images security
  • Registry security
  • Orchestrators security
  • Container engine security

Container images security

To implement container image security, you need:

  • Vulnerability scanning – scanning images as part of your build process makes a lot of sense for identifying known vulnerabilities, and failing your builds when these vulnerabilities are encountered is a must for keeping your containers secure
  • Minimal base image – containers are lightweight, and they are lightweight for security reasons too. By using a minimal base image that includes only the necessary components for your applications to run, you reduce the potential attack surface by eliminating unnecessary packages, thus eliminating potential vulnerabilities
  • Image signing – you should only use images from a known source

Registry security

Container registries are responsible for storing container images after they are built. Securing the registry involves:

  • Implementing least privilege access – strict access control to manage who can push and pull images will solve many potential security issues
  • Regular registry scan – scanning images shouldn’t be done only when you are building the image but also throughout the lifecycle of that image
  • Protect against interception – you should encrypt your images during transport to and from the registry

Orchestrators security

Nowadays, containers are used mostly with an orchestrator. The most popular orchestrator is Kubernetes, and there are many things you can secure for your orchestrator:

  • RBAC – ensure users get the least privileged access
  • Network policies – implement network policies to control the traffic flow
  • Secrets management – manage secrets securely, rotate them, and have controlled access to them

Container engine security

The container engine is responsible for running containers on the host machine, and securing it involves:

  • Implementing runtime security – monitor and protect containers in real-time (anomaly detection, blocking suspicious events)
  • Host security – ensure users get the least privileged access to the host that runs the containers
  • Resource limitations – set resource limits to prevent distributed denial of service (DDoS) attacks

Container security common challenges and risks

The most common security challenges, alongside their risks and some potential solutions, are:

Common security challenge Risks Solution
Vulnerabilities in container images Attackers exploiting these can gain access to other parts of the system Implement security vulnerability scanning as part of your build process and your registry
Insecure container registries Unauthorized access Ensure the least privileged access and encrypt your images during transport
Misconfigurations Create vulnerabilities that can be exploited to get unauthorized access or service unavailability Ensure the build and deployment process of your images and containers are fairly tested & regular audits
Secrets management Data breaches Use dedicated secrets management tools and encrypt secrets in transit and at rest
Network issues Without appropriate segmentation, malicious traffic can increase an attack’s surface Implement network policies and use service meshes
The ephemeral nature of containers Makes it hard to detect issues in real-time Adopt runtime security solutions

Container security best practices and solutions

The most important container security best practices:

  1. Use trusted base images
  2. Keep images up to date
  3. Reduce the attack surface
  4. Limit container privileges
  5. Implement access controls
  6. Scan images for vulnerabilities
  7. Implement network security
  8. Monitor container activity
  9. Compliance and auditing
  10. Train your team

Let’s look at each area in turn, and some suggested solutions!

You can also check out how Spacelift brings the benefits of CI/CD to infrastructure management. You can use Spacelift to deploy changes to Kubernetes with GitOps while benefiting from robust security policies and automated compliance checks. Spacelift also works with other infrastructure as code (IaC) providers, so you can use similar techniques to manage every component of your stack.

1. Use trusted base images

Avoid using images from unknown sources, as they may contain malicious code. When possible, use container images that are provided by the official repositories of the software vendor or organization. These images are usually tested, verified, and maintained by the vendor or organization, ensuring their quality and security.

Always check the image’s source code repository, its maintainers, and its community to ensure that it is reliable and trustworthy. Make sure signed images are used, which use cryptographic signatures to ensure that the image has not been tampered with or modified since it was signed by the author. Signed images can help protect against malicious attacks and supply chain attacks.

2. Keep images up to date

Keep your container images up to date with the latest security patches and software updates to prevent known vulnerabilities from being exploited. Establish a regular schedule for updating your container images. Depending on the frequency of updates released by the software vendor or organization, consider scheduling updates weekly, bi-weekly, or monthly.

You can also consider automating the updates. To help with this, always choose container image registries that support automatic updates and rollbacks, making it easier to manage image updates at scale.

If your cluster is running on Amazon EKS, you can use managed node groups to automatically manage and update the underlying EC2 instances used to run your Kubernetes workloads. This helps to ensure that your nodes are running the latest security updates.

3. Reduce the attack surface

Use minimalistic images with only the necessary software installed. This reduces the attack surface and minimizes the potential for security breaches. Using lightweight images will also increase your container start-up time and performance.

There are many images out there that aim to meet this goal, each with its own strengths and weaknesses, depending on your specific use case.

Popular choices include:

  1. Alpine Linux: A lightweight Linux distribution that weighs in at only 5 MB. It’s a popular choice for Docker images due to its small size.
  2. Ubuntu Minimal: This is a minimalistic version of Ubuntu that’s designed for use in containers. It includes only the essential packages needed to run Ubuntu and weighs in at around 50 MB.
  3. Scratch: This is a special Docker image that’s essentially an empty container. It doesn’t have a package manager or any system utilities, but it can be useful for building truly minimalistic images from scratch.
  4. BusyBox: This is a minimalistic Linux distribution that includes a variety of common Unix utilities, such as ls, cp, and grep. It’s only about 2 MB, making it a good choice for small Docker images.
  5. Tiny Core Linux: Another lightweight distribution popular for container images. It’s only 16 MB in size and includes a minimalistic graphical user interface.

4. Limit container privileges

Containers should be run with minimal privileges, such as non-root user mode, and limit their access to the host system and other containers. Running containers as root users can give the containers too much power and access to the host system.

User namespaces can be used to map container users to different users on the host system through UID mapping.

By using UID mapping with user namespaces, you can isolate container users from the host system while still allowing them to have access to the host’s resources. This can be useful for running containers with different privilege levels and reducing the risk of privilege escalation attacks.

  1. Create a user called jack with a UID of 9999.
sudo useradd -u 9999 jack
  1. Create a new user namespace for the container using the “newuidmap” and “newgidmap” utilities. This will map the container’s root user (UID 0) to the host user “jack” (UID 9999).
sudo unshare --user
echo "0 $(id -u) 1" | sudo tee /proc/self/uid_map
echo "0 $(id -g) 1" | sudo tee /proc/self/gid_map
exit
  1. Start the container using the new user namespace using the userns=host option, which tells Docker to use the host user namespace for the container.
docker run --rm -it --userns=host alpine sh
  1. Verify that the container’s root user is mapped to the host user “jack” using the id command:
su jack
id

Note: Popular tools like AppArmor or SELinux can also be used to restrict container access to sensitive system resources. Secure computing mode (or seccomp) is a Linux kernel feature that can be used to limit the system calls that containers can make.

5. Implement access controls

Implement access controls and limit container access to only those who need it. Use authentication and authorization mechanisms to ensure that only authorized users can access the containers.

Kubernetes RBAC (Role-Based Access Control) is a security feature that allows administrators to control access to resources in a Kubernetes cluster based on a user’s role or job function. With RBAC, administrators can define roles, role bindings, and service accounts that grant specific permissions to users or groups of users.

The key components of Kubernetes RBAC:

  1. Roles: A role is a set of permissions that define what actions a user or group of users can perform on a specific set of resources. In this example, a role named pod-reader is defined with permissions to get, watch, and list pods in the my-namespace namespace.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: my-namespace
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
  1. Role Bindings: A role binding is a mapping between a role and one or more users, groups of users, or service accounts. In this example, a role binding is created that binds the pod-reader role to the user alice in the my-namespace namespace, and grants alice permissions to read pods in the my-namespace namespace.
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-pods
  namespace: my-namespace
subjects:
- kind: User
  name: alice
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
  1. Service Accounts: A service account is an identity that is used by a pod to access other Kubernetes resources. Service accounts can be granted specific permissions using role bindings. The example below shows how to create a service account called my-service-accountand how to use it to create role bindings to grant permissions to access resources in a namespace.
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-service-account
  namespace: my-namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: my-role-binding
  namespace: my-namespace
roleRef:
  kind: Role
  name: my-role
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: my-service-account

If your cluster is running on AKS, you can use Azure Active Directory (AAD) integration to control access to AKS resources and enforce strong authentication and authorization policies. If you are using Amazon EKS, use IAM.

6. Scan images for vulnerabilities

Use a container image scanning tool to detect vulnerabilities and ensure that your images are free from security risks before deployment. These tools can be integrated into the CI/CD pipeline to automatically scan images as they are built and deployed, ensuring that only secure images are used in production environments.

Popular tools include:

  1. Astra Security: Astra Security’s vulnerability scanner provides automated and manual security scans for cloud-based containers. The tool conducts 9300+ tests to detect vulnerabilities in container configurations and images during its runtime.
  2. Trivy: Trivy is an open-source image scanner that is designed to be lightweight and fast. The tool checks container images for known vulnerabilities in both operating system packages and application dependencies.
  3. Anchore Engine: Anchore Engine is an open-source tool that provides container image analysis and policy-based evaluations. The tool checks container images against customizable policies and can provide notifications when policy violations are detected.
  4. Aqua Security: Aqua Security provides a container security platform that includes an image scanning tool. The tool checks container images for known vulnerabilities, malware, and other security threats.
  5. Clair: Clair is an open-source container image scanner that is integrated with many popular container registries like Docker Hub, Quay, and Google Container Registry. The tool checks container images for known vulnerabilities and provides detailed reports on any issues found.
  6. Sysdig Secure: Sysdig Secure is a container security platform that includes an image scanning tool. The tool checks container images for vulnerabilities, malware, and configuration issues.

7. Implement network security

Implement network security controls to prevent unauthorized access to your containers. Use network segmentation to isolate containers and limit network access to only the necessary services.

Kubernetes network policies can be used to define rules for incoming and outgoing traffic to pods in your cluster by limiting traffic to specific ports and protocols and restricting traffic to specific pods based on labels. TLS encryption can be implemented for communication between pods, and use SSL/TLS certificates can be used to secure communication between the API server and other components in your cluster. Load balancers and other tools can be used to limit ingress traffic to your cluster.

You can also implement a service mesh like Istio or Linkerd to secure traffic between microservices in your cluster. Service meshes provide a centralized control plane for traffic management and can help secure communication between services using mTLS encryption.

Don’t forget to secure your network if using a cloud provider to host your cluster as a PaaS service, for example with Azure Kubernetes Service (AKS), you should use network security groups (NSGs) to control inbound and outbound traffic to and from AKS and restrict traffic to only the necessary ports and protocols. For Amazon Elastic Kubernetes Service (EKS), use VPC security groups.

8. Monitor container activity

Monitor logs and metrics to detect anomalies and respond to security incidents. Implement container monitoring to detect suspicious activity and potential security breaches. Security tools like cAdvisor or kube-state-metrics can be used to collect resource usage metrics.

Popular container monitoring tools like Prometheus and Grafana can be used to monitor the performance and activity of containers running in your cluster. These tools can help you detect issues with resource usage, application performance, and other important metrics.

Use network monitoring tools like Wireshark or tcpdump to capture and analyze network traffic between containers. This can help you detect potential security threats or performance issues.

If you are using AKS, you can use Azure Monitor to monitor resources and detect security issues and Azure Log Analytics to collect and analyze log data from AKS resources. For Amazon EKS, use Amazon CloudWatch to monitor and Amazon CloudTrail for logging.

9. Compliance and auditing

Addressing compliance and auditing in a containerized environment can be hard to do. These processes should incorporate both technological solutions and process-oriented strategies. Due to their ephemeral nature and the dynamic environments in which they operate, containers have unique challenges in ensuring compliance with regulatory standards and conducting effective audits.

You should implement automated compliance checks using automated tools (such as Chef InSpec or Aqua Security) against the CIS (Center for Internet Security) benchmark, HIPAA (Health Insurance Portability and Accountability Act), and GDPR (General Data Protection Regulation) requirements. These automated tools can provide real-time alerts and reports on compliance status.

10. Train your team

Lastly, you should take steps to educate your team on container security best practices and train them on how to identify and respond to security threats.

Following container security best practices should be part of the training after an overview of containerization and related security risks. Hands-on training, regular security assessments, and continuous learning should help keep your team up to date with the latest security trends and techniques.

Key Points

Effective container security involves a holistic approach that encompasses not only the containers themselves but also the underlying infrastructure, networks, and user access controls. By implementing robust container security practices, you can ensure that your applications are protected from potential threats and vulnerabilities.

The Most Flexible CI/CD Automation Tool

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

Start free trial

The Practitioner’s Guide to Scaling Infrastructure as Code

Transform your IaC management to scale

securely, efficiently, and productively

into the future.

ebook global banner
Share your data and download the guide