AWS Cost Optimization: Strategies, Best Practices, and Tools

Cloud adoption is a de facto standard today. Key reasons for this are convenience, the facility to scale up and scale down resources on demand, and the ability to pay only for the resources you consume.

Although cloud computing has many benefits, enterprises sometimes end up paying more for their services due to a lack of appropriate planning, architecture, and optimization and improper monitoring of resource consumption.

In this post, we explore AWS cost optimization strategies and best practices for cost-efficient utilization of cloud resources.

We will cover:

1. Overview of cloud pricing models
2. General cloud cost optimization strategies
3. AWS design principles for cost optimization
4. Practical tips for cost optimization
5. Cost optimization best practices
6. AWS cost management tools

In general, IAAS cloud pricing models are based on three fundamental aspects — compute, storage/database, and network utilization. For example, AWS varies its services to optimize costs based on business needs.

1. Compute:  AWS offers Amazon Elastic Compute Cloud (EC2), Amazon Container Services (ECS), Amazon Kubernetes Services (EKS), AWS lambda, etc. Attributes like availability, managed/shared/dedicated tenancy, and workload nature differ for each of these services., potentially affecting costs.
   • Example compute cost calculation: $ per seconds /minutes/hours/…/days
2. Storage/databases: AWS offers Amazon Simple Storage Service (S3), Amazon Elastic Block Storage (EBS),  Amazon Elastic File System (EFS), Amazon Glacier, etc., as storage solutions. Database services include Amazon RDS, Amazon DynamoDB, Aurora, etc. This flexibility allows customers to move data to less frequently accessed storage classes to save costs.
   • Example storage/database cost calculation: $ per Mb/Gb/Tb
3. Network: Costs associated with network bandwidth consumption mainly depend on two aspects — the speed of the connection and the amount of data transferred (inbound/outbound).
   • Example network consumption cost calculation: $ per data transfer/ No. of API requests

The examples above provide a high-level overview of how cost calculations are made based on the three core infrastructure components.

Each service is associated with a complex pricing model, making it difficult to arrive at a final figure for the costs associated with any given cloud architecture.

Thankfully, AWS provides a pricing calculator to create estimates for each service depending on the customization options chosen.

Effectively managing cloud costs doesn’t just save money — it ensures your infrastructure runs efficiently. A robust cloud cost optimization strategy typically revolves around four main pillars:

1. Right-sizing resources: Adjust resources to perfectly match your workload needs.

• • Ensure that cloud resources including virtual machines, storage, and databases are appropriately sized to meet current needs.
  • Analyze usage patterns and adjust resources to avoid over-provisioning or under-utilization.
  • Scale resources dynamically to match workload requirements.

2. Optimizing storage: Use the right type of storage for your data and regularly review unused or infrequently accessed files.

• • Use tiered storage to manage costs (e.g., move infrequently accessed data to cheaper storage tiers).
  • Compress and deduplicate data to save space and cost.
  • Clean up unused or redundant data regularly.
  • Leverage lifecycle policies for automated storage transitions.

3. Leveraging reserved instances and savings plans: Commit to long-term resource use for lower rates.

• • Evaluate your consistent workloads to determine eligibility.
  • Choose the right Savings Plan or Reserved Instance model (e.g., 1-year, 3-year plans).
  • Balance commitments with flexibility to adapt to future needs.

4. Monitoring and automation: Continuously track usage and automate optimizations.

• • Monitor resource usage and expenses with tools like AWS Cost Explorer, Azure Cost Management, or Google Cloud Cost Management.
  • Automate policies to turn off idle resources and scale workloads during off-peak times.
  • Use cost anomaly alerts to detect and address overspending quickly.

Note that this is not an exhaustive list of principles but a suggested approach for cloud cost optimization.

AWS’ well-architected framework provides a set of best practices and guidelines for designing and operating reliable, secure, efficient, and cost-effective systems in the cloud. Cost optimization is one of the five pillars of this framework. It aims to achieve the desired outcomes with justified costs.

In this section, we discuss design principles that guide us in optimizing costs.

1. Monitor resource consumption
2. Adopt a consumption-based pricing model
3. Use the right compute instances
4. Use the right storage classes
5. Leverage AWS Cost Explorer for analysis

1. Monitor resource consumption

Tracking and monitoring always precede optimization efforts. Monitoring provides context and understanding for why certain costs need to be optimized, shedding light on overspending. Several tools provided by AWS and other third-party services help uncover unnecessary expenditures:

1. CloudWatch Metrics provides various metrics and the ability to set alarms when a certain threshold is met. With CloudWatch Metrics, we monitor compute resource consumption, database storage capacities, and various other metrics.
2. AWS Cost Explorer provides a detailed analysis of cloud spend by service.
3. Third-party tools that monitor AWS cloud costs provide reporting on cloud resource consumption and price associated with it.

2. Adopt a consumption-based pricing model

Instead of reserving resources to serve uneven or occasional traffic, adopt consumption-based resource allocation by leveraging auto-scaling capabilities. This helps reduce costs by avoiding overprovisioning of resources and paying for unused resources.

Cloud-native services are better at supporting cost optimization and finer control over cloud resource utilization.

For example, AWS Fargate or Lambda functions create additional compute instances only when they are needed and in a quicker way than EC2 instances. 

3. Use the right compute instances

It is possible to classify workloads as critical and non-critical. Provisioning compute instances with general configuration on shared resources for less critical workloads may make sense. Limiting the provisioning of dedicated servers to critical workloads only helps optimize costs.

The steps to identify and provision appropriate compute resources based on utilization and criticality are summarized below:

1. Analyze the resource usage with AWS CloudWatch to identify which resources are underutilized.
2. Identify optimal resource levels that will suffice your business needs.
3. Resize resources either by modifying resources or adjusting the number of running instances.
4. Automate the above process to maintain the right sizing of the resources. 

The table below summarizes various types of EC2 instances and associated features.

Additionally, choosing Amazon’s Savings plan on EC2 can give you significant cost savings (72%) in exchange for a commitment of one to three years.

4. Use the right storage classes

AWS EC2 instances often use Elastic Block Storage (EBS) to persist data. Pricing of volumes usually depends on the size and type of EBS volumes. EBS pricing depends on performance characteristics, size, I/O, region, etc. The table below summarizes various types of EBS volumes along with the use case.

Similarly, various classes of S3 storage offer lower prices by compromising on aspects like availability, redundancy, and retrieval of historical data. Depending on consumption patterns, data can be moved to colder storage solutions, which, in turn, reduces storage costs over time.

5. Leverage AWS Cost Explorer for analysis

AWS Cost Explorer is a cost analysis tool that displays cost division based on service in a granular and detailed manner. Use Cost Explorer to track spending over time and identify areas for cost optimization.

Generate custom reports to provide insights into cost drivers, usage patterns, and trends. As a best practice, it is always suggested to use tags while provisioning various cloud resources for billing purposes. Cost Explorer is a tool that culminates those tagging efforts into meaningful reports and actions.

This section includes tips for optimizing costs for some commonly used services.

Optimizing AWS S3 cost

Amazon S3 is an object storage service that facilitates use cases like data backup and restore, data archival, data lake, and other enterprise application storage. S3 stores data in buckets similar to the directories in your local system. The actual data you store in S3 as a fundamental unit is called an object, which is similar to files. 

Storage classes are one of the primary determinants of Amazon S3 pricing. Each object is associated with a storage class. It is important to identify which storage class should be assigned to the object.

If you’re unsure about the data access pattern and volume of the data, then use either S3 analytics or S3 storage lens to get insights into the usage. This will help you identify the data that should be in a particular storage class and when.

You can also leverage the S3 lifecycle rules to automatically transition your data based on these insights.

Optimizing Amazon EBS cost

Here are some practical tips for optimizing EBS usage:

1. Unattached volumes – After terminating EC2 instances, you might not need their attached volumes. Unattached volumes incur the same cost as those attached to an EC2 instance. Best practice is to take the volume’s backup as a snapshot and then remove the volume. 
2. Snapshot policies — If the EBS volume is hosting a database, it creates many incremental snapshots. These snapshots may incur substantial AWS S3 costs. Instead, implement a retention policy for deleting the older snapshots to save S3 costs.

Optimizing data transfer charges

AWS services like Amazon EC2, Amazon RDS, and Amazon S3 do not offer free inbound data transfer. However, outbound data transfer is chargeable. Always monitor the amount of data transferred from these resources to the public internet and limit it where it makes sense.

If the outbound data is static and repetitive, consider using Amazon CloudFront. This is a CDN service designed for caching and serving static content, reducing outbound costs compared with EC2 machines.

AWS also charges for data transfer between regions and availability zones. These are important considerations when designing and architecting systems to be deployed in the cloud.

Optimizing RDS costs

AWS RDS costs are optimized based on factors including:

1. Right-sizing the instance by choosing appropriate CPU, memory, and storage requirements
2. Using read replicas helps offload read traffic from primary database instances.
3. Writing efficient queries and improving database schemas also help in the processing and retrieval of data sets
4. Using reserved instances to get discounts wherever possible
5. Using database engine features appropriately, like backups, AZ, and read replicas

Optimizing DynamoDB costs

AWS DynamoDB costs can be optimized based on factors including: 

1. Choosing the appropriate capacity (On-demand or provisioned, or reserved capacity)
2. Choosing the right table class (Standard or Standard infrequent access)
3. Using the TLL feature to automatically delete expired data
4. Reviewing secondary index usage and deleting any unused or unnecessary indexes
5. Reviewing backup retention periods

Adopting IaC

Infrastructure as code (IaC) is a way to manage and provision resources using code. Terraform, Pulumi, and AWS CDK are some examples of IAC. Adopting IAC for managing cloud resources will have several benefits, some of which are outlined below:

1. IaC enables you to automate the provisioning, configuration, and management of our resources. This automation can help you reduce the time and effort required to manage your infrastructure, reducing costs..
2. IaC enables versioning of infrastructure.
3. Use templates to standardize. Ensure that infrastructure is consistent across environments to reduce errors and increase efficiency.

Leveraging serverless architecture

Serverless architecture allows applications to run while the cloud provider manages the underlying infrastructure. AWS cloud services typically used in serverless designs are AWS Lambda, AWS API gateway, AWS DynamoDB, AWS SNS, AWS step functions, AWS S3, and AWS Cognito.

Leveraging serverless reduces operational costs. Furthermore, you are charged only as long as it takes for your code to execute, providing a more cost-effective pricing model than paying for idle instances or containers.

However, serverless is not always a superior alternative to using EC2 instances or containers. It requires you to rethink your infrastructure and redesign solutions.

These best practices enable you to reduce your AWS costs while maintaining optimal performance and scalability:

1. Identify the type of EC2 instance based on the requirement. Provision reserved instances for production (or sub-production) environments.
2. Opt for the AWS savings plan, which also provides great discounts (72%) on Amazon EC2, Amazon Fargate, and Amazon Lambda.
3. Shutting down or terminating, dev, and testing EC2 machines when not in use (after working hours and on weekends).
4. Automate right-sizing of resources by leveraging auto-scaling and CloudWatch events.
5. Use appropriate S3 storage classes for the objects. Use S3 lifecycle rules to transition the rarely accessed data to lower-cost tiers.
6. Delete very old snapshots of the EBS. You should have the lifecycle for snapshots.
7. Always destroy unattached EBS volumes. Take snapshots before destroying them.
8. Release unattached elastic IP addresses after terminating EC2 instances.
9. Make cloud optimization part of your DevOps lifecycle.
10. Use serverless solutions for low-traffic applications and short tasks.
11. Avoid data transfer between AZs and regions. Keep services in the same region and AZ.
12. Provision Amazon CloudFront distributions for repetitive and static outbound data.

These tools are used for cost optimization of AWS resources:

1. AWS Trusted Advisor – a web-based service provided by AWS that offers real-time guidance to help you optimize your AWS environment according to AWS best practices. It evaluates the AWS account and provides a way to optimize infrastructure, improve security, reduce costs, and monitor service quotas.
2. AWS Cost Anomaly Detection – a feature within the AWS Cost Management suite that utilizes machine learning to monitor and identify unexpected increases in your AWS spending. It can identify unusual spending trends and their underlying causes, empowering teams to respond promptly.
3. AWS Cost Explorer – a user-friendly tool that allows you to visualize, understand, and manage your AWS costs and usage over time. It provides an interactive dashboard to visualize and manage AWS resources, enabling users to view, analyze, and optimize infrastructure for cost and usage.
4. AWS Compute Optimizer – a service that leverages machine learning to analyze the configuration and utilization metrics of your AWS resources, such as Amazon EC2 instances, Auto Scaling groups, Amazon EBS volumes, AWS Lambda functions, and Amazon ECS services on AWS Fargate. It provides recommendations to optimize resource usage based on historical usage patterns.
5. AWS Pricing Calculator – a free, web-based tool that enables you to estimate the costs of AWS services for your planned use cases by specifying the service information in detail.
6. AWS Auto Scaling – a service that monitors your applications and automatically adjusts capacity to maintain steady, predictable performance at the lowest possible cost. It allows you to automatically adjust the capacity of AWS resources based on the demand of your application providing cost optimization opportunities.
7. AWS CloudWatch – a monitoring and observability service that captures and presents real-time logs, metrics, and event data on automated dashboards to optimize infrastructure and application management.
8. AWS Lambda Power Tuning – an open-source tool designed to help you visualize and fine-tune the memory and power configurations of your AWS Lambda functions. It helps optimize the performance and cost-effectiveness of AWS Lambda functions by using machine learning to analyze and optimize a Lambda function’s configuration parameters, such as memory allocation, timeout settings, and concurrency levels.
9. Infracost – a third-party open-source tool that provides cloud cost estimates for infrastructure as code (IaC) projects, particularly those using Terraform. This is a key insight from the cost optimization perspective. It is integrated with Spacelift and is available in the free tier!

Spacelift is an infrastructure orchestration platform that increases your infrastructure deployment speed without sacrificing control. With Spacelift, you can provision, configure, and govern with one or more automated workflows that orchestrate Terraform, OpenTofu, Terragrunt, Pulumi, CloudFormation, Ansible, and Kubernetes. 

You don’t need to define all the prerequisite steps for installing and configuring the infrastructure tool you are using, nor the deployment and security steps, as they are all available in the default workflow.

With Spacelift, you get:

• Integration with Infracost
• Policies to control what kind of resources engineers can create, what parameters they can have, how many approvals you need for a run, what kind of task you execute, what happens when a pull request is open, and where to send your notifications
• Stack dependencies to build multi-infrastructure automation workflows with dependencies, having the ability to build a workflow that, for example, generates your ec2 instances using Terraform and combines it with Ansible to configure them
• Self-service infrastructure via Blueprints, or Spacelift’s Kubernetes operator, enabling your developers to do what matters – developing application code while not sacrificing control
• Creature comforts such as contexts (reusable containers for your environment variables, files, and hooks), and the ability to run arbitrary code
• Drift detection and optional remediation

Spacelift was built with DevOps/platform engineers in mind, but it developed to become the go-to platform for software engineers, too, because they can increase their velocity with self-service infrastructure that implements all your organization’s guardrails. It greatly enhances collaboration among engineers, offering them a central location to make infrastructure-related decisions.

If you want to learn more about what you can do with Spacelift, check out this article.

Does your organization have extra compliance concerns? Here you can learn more about self-hosting Spacelift in AWS, to ensure your organization’s compliance, control ingress, egress, internal traffic, and certificates, and have the flexibility to run it within GovCloud.

Enterprises are concerned about spending excessively on AWS resources. In this post, we covered various scenarios and discussed various approaches for limiting expenditure on AWS by leveraging the flexibility of resource consumption and associated pricing that AWS provides.

We also explored some of the best practices and AWS cost optimization tools that help with cloud cost management. The points we discussed are not exhaustive, and every organization’s use cases differ vastly. It is important to observe these principles continuously to optimize infrastructure costs.

If you are interested in a product that can orchestrate provisioning, configuration, and governance, across your infrastructure, Spacelift is the answer. Create a free account with Spacelift today, or book a demo with one of our engineers.