Patterns That Set Infrastructure Automation Leaders Apart

With developer self-service, developers request new cloud infrastructure and have it provisioned by automation workflows.

Self-service provisioning is often combined with an internal developer platform (IDP), which is the front end of the underlying automation platform(s).

Why does focusing on developer self-service make your organization a leader in cloud infrastructure automation?

The answer to this question is two-fold:

1. Speed:  Self-service workflows enable developers to work fast. They can request the cloud infrastructure they need without waiting on slow approval processes.
2. Control: Self-service workflows give the platform team full control over the provisioned cloud infrastructure. Compliance, security, and regulatory requirements can be built into the self-service offerings from the start. The platform team can allow the developers to configure variable parts of the infrastructure, while restricting other parameters to approved and secure defaults..

The infrastructure automation report highlighted the ongoing conflict between speed and control. To some extent, with self-service workflow capabilities, you can eliminate the speed-control paradox. Correct implementation of self-service offers speed and control in perfect harmony.

The infrastructure automation report highlighted the ongoing conflict between speed and control. Self-service workflow capabilities help to eliminate the speed-control paradox. Correct implementation of self-service offers speed and control in perfect harmony.

How can an organization focus on developer self-service yet still not be a leader?

Even if you implement developer self-service, you can fall into traps that prevent the pattern from making your organization a leader.

Some of these traps are:

• It’s too complex: The self-service modules you offer require manual intervention and intimate knowledge of the underlying resources.
• It doesn’t solve any real problems: The self-service modules you offer are easily replaced by custom IaC templates and do not offer any additional value. Using a self-service workflow should be faster for the team than building the cloud infrastructure themselves.
• It offers the wrong level of abstraction: Self-service is only good if the self-service modules offer the right level of abstraction and customizability. These are tough requirements to get right. Ideally, your self-service modules should fulfill 80% of the use cases they are intended for out of the box with no further customization. The other 20% should be addressed by customization options for advanced use cases.

To get your self-service offerings right, you should start small and listen to what the developers (your customers) require.

Integrating security and compliance early is often much less costly than doing so later in the process. Strengthening your cloud infrastructure environment while recovering from a severe security incident is costlier than focusing on security from day one to avoid major security incidents.

Security and compliance have a reputation for slowing development, favoring control in the speed-control paradox. However, focusing on security and compliance does not have to slow you down.

A tradeoff to keep in mind when discussing compliance is that not fulfilling certain compliance frameworks can negatively affect your organization, even to the point of putting it out of business.

Why does integrating security and compliance early make your organization a leader in cloud infrastructure automation?

Having a secure and compliant foundation allows you to focus on business value and building the products and solutions that your end users want.

Security and compliance are rarely part of an organization’s core business, but without them, you might not be able to do business at all. However, security and compliance are part of the platform team’s core business.

Focusing on security and compliance and building it into your workflows from the start makes your developers happier, allowing them to focus on what differentiates your organization. Abstracting away the issue of compliance and security for your developers puts you in a very good position.

How can an organization focus on integrating security and compliance early yet still not be a leader?

Some traps you can run into when focusing on integrating security and compliance include:

• Overdoing it: Security is important, but you need to know where to draw the line. One example is adding manual approval gates at every stage of automation. This could initially appear to strengthen your security and compliance stance, but it is counterproductive and slow. Manual approval decisions and validation can often be replaced by automation with far greater success.
• Putting too much responsibility on your developers: If you introduce numerous policies to ensure your infrastructure fulfills the required regulatory frameworks,  without explaining them to the developers,  your development teams will grind to a halt. Ideally, governance should be built in (see the discussion on self-service above), but at least it should be easy to consume and understand. It is up to the platform team to provide a streamlined experience for the development teams.
• Bolting on security and compliance too late: It is common for security and compliance to be added late. Your teams may have been working with cloud infrastructure automation for a while, but the game changes when you introduce new security standards and compliance frameworks that break half of your team’s automation.

A platform engineering team is a dedicated team working to establish a platform for cloud infrastructure and application development for the rest of the organization. The platform is the foundation supporting the rest of your business.

This practice has been gaining traction over the past few years, and it is now common for organizations to have some form of dedicated platform team.

Why does focusing on adopting a platform team approach make your organization a leader in cloud infrastructure automation?

The platform team’s mission is to enable the organization to build and deploy applications running on cloud infrastructure. They should reduce the friction that developers encounter with anything that is not part of the developers’ core business.

Using a dedicated team for this instead of letting every other team reinvent the wheel streamlines how the business develops new features and applications. Developers can concentrate on producing value for the end customers, as the platform team concentrates on providing value to the developers.

How can you adopt a platform team approach yet still not be a leader?

The answer to this question has many factors.

First, if you adopt a platform team approach without a clear platform engineering strategy, chances are you will not really build anything useful for the organization.

Every part of the organization must be on board for the platform journey. A common scenario is introducing a platform team that another team might consider to be forcing certain ways of working on the organization. The result is friction between the platform team and every other development team.

The platform must be useful for developers, easy to use, and the preferred option when developers launch new products in the cloud. If this is not true, developers will likely detour around the platform and implement their own custom solutions.

Whatever the reason is, your platform approach is not working. This is a bad situation to be in, and the platform is not giving you the benefits that the leaders see from their approach.

For organizations starting out in the cloud, the cost is often hard to predict. Two common cost categories that surprise users are log ingestion and network egress traffic.

Without a clear strategy and cost tracking, your cloud costs are likely to be larger than you had hoped.

FinOps is gaining traction in organizations. It is the practice of focusing on cost optimization and working towards efficient cloud spending with no or little cloud waste.

Why does focusing on prioritizing cost optimization make your organization a leader in cloud infrastructure automation?

Focusing on spending your cloud budget where it has the biggest impact allows you to maximize the return on investment (ROI). This is impossible without having a developed FinOps approach where you track and understand your costs and continuously work to improve your financial situation.

If you can reduce cloud waste, you have a bigger budget for important resources that increase your speed.

Managing costs falls under the control part of the speed-control paradox. However, by focusing on costs, you will gain speed in the long run due to better ROI and a larger budget to spend on things that really matter.

How can you prioritize cost optimization yet still not be a leader?

The hard fact is, the cloud costs money.

Certain aspects of the cloud should not be optimized for cost. One obvious area is cloud security.

If you take your FinOps approach so far that it starts to negatively impact your cloud infrastructure automation, your cost savings will be useless. A concrete example is using build agents with inadequate resources (CPU and memory), so that builds and tests are slow, negatively impacting the speed of development teams.

Generally, when cost optimization starts to introduce bad choices in your cloud infrastructure,  the focus on costs is working against you rather than making your organization a leader.

Siloed automation is what you get when every team reinvents the wheel on its own. Every team decides on which tools to use, which processes to automate, and how to do it.

Giving developers 100% freedom does not equal speed. You will lose speed because you need to spend so much time solving problems that have already been solved many times before, even within your own organization.

Why does focusing on avoiding siloed automation make your organization a leader in cloud infrastructure automation?

Not solving the same issue repeatedly in an organization means you can solve it once package the solution into a reusable component, and offer it through self-service. This increases speed and allows you to build in governance and control.

As mentioned, security and control are probably not part of your core business. The same is true for solving automation issues. Is optimizing the build step for your Go applications important for your end users? Probably not. The same goes for evaluating different scanners for infrastructure as code to determine which one to use.

When developers can focus on their core business, they are happier and more productive where it matters.

How can you avoid siloed automation yet still not be a leader?

If you are building inefficient, error-prone automation, centralizing the automation work will not benefit you. 

Any type of centralized approach done badly is worse than a siloed approach. In the latter, at least some teams will probably do a good job and reach their goals, whereas no team will be in a good position with the former approach.

It doesn’t have to be speed or control: you can have both. Leaders in cloud infrastructure automation follow a few patterns that set them apart. However, implementing these patterns blindly does not make your organization a leader in cloud automation.

You need to have a strategy for automation in your organization and work towards reducing friction, increasing developer speed, and building governance and control from the start without bolting it on at the end.