DevSecOps: Making Security Central To Your DevOps Pipeline

In this article, we’re going to take a closer look at a very important part of the DevOps pipeline: security.

Because of how important security is to DevOps pipelines, many have taken to calling it DevSecOps instead. What exactly is it, and how can you properly implement it in your organization? Let’s take a look.

Table of contents:

1. What is DevSecOps?
2. Advantages of DevSecOps
3. Organizational Culture
4. The DevSecOps Lifecycle
5. Implementing the Proper Tools and Processes

The point of DevOps is to increase your team’s productivity by breaking down silos between development teams and operations teams.

The point of DevSecOps, therefore, is to implement security best practices while still maintaining increased team productivity by breaking down silos between teams.

This is important because most people view security as an annoyance and as something that adds extra friction to the development and deployment process. Instead of accepting that and putting security on the back burner, DevSecOps brings security back to the forefront by saying “you know what, security is critical and must be part of our process, so let’s make it fit in as painlessly as possible.”

Shift-left security

To make security more impactful without adding much friction, we need to include it from the very beginning. The traditional approach has been to plan, develop, and push changes forward. Only then would a security team become involved. 

There are many issues with this approach.

One issue is that the security team will not have access to most of the context. Lack of context results in questions like these being asked without an easy way to gather answers and typically way too late in the process:

• Why is this being developed the way that it is?
• Who made that decision?
• Are they aware it will have an impact on other parts of the system?

Another issue is that when the security team finds problems, the pipeline has to go all the way back to the beginning. That causes delays and frustration, especially if the development or infrastructure team has already moved on to another task.

Instead, with DevSecOps, we want to include security from the very beginning so that problems are found as early in the process as possible: closest to the decision makers and to the context.

One way we can do this is by including security team members during planning and then again during product demonstrations in between each development interval. This helps threefold:

1. It helps the security team understand business goals for the implementation
2. It involves a team that will be impacted during the development instead of after
3. The security team can provide guidance and feedback as features get created

This approach provides security expertise earlier in the development process and reduces the amount of time before issues get discovered.

Another way we can do this is by tracking open security issues in the same tracking systems that development and operations teams use. The traditional approach has been to keep security issues in separate tracking systems, which means Dev and Ops don’t have visibility (and vice versa). By having all open issues (from Dev, Sec, and Ops) in a central location, everyone can see what the priorities are, and they can collaborate.

If we envision our pipeline going from left to right — left being the starting point and right being production — by moving security at the very beginning, we are shifting security left. That’s why this is often referred to as a “shift-left” security strategy.

Ask any developer, infrastructure engineer, or security engineer, whether they’d prefer someone to find issues with their code, infrastructure, etc… while they’re still working on it versus after they’ve pushed it through and moved on to another task and the answer will be the same: show me issues as early as possible so that I can quickly fix them and move on to something else.

It’s much easier and less painful to fix issues while the project is fresh on your mind than after you’ve already switched to a different context. It’s much more efficient.

Let’s take a closer look at how those efficiencies can benefit the organization and team.

The reason that security is so important to include from day one is that it will help:

• Reduce risk and legal liability
• Identify bugs and vulnerabilities earlier in the process
• Make security more manageable

Reducing risk and legal liability

When it comes to reducing risk and legal liability, we’re not just talking about getting sued by shareholders or customers when there’s a costly breach. We’re also talking about being in compliance with the law.

Depending on your business, you may have to comply with GDPR law depending on certain situations. Or, you may have to handle health-related records, which means you may need to comply with HIPAA or other important laws.

If you don’t comply, it could cost your organization a massive amount of money in fees.

Having the proper tools and processes in place can help prevent modifications from going out that would result in violations.

It can also help identify compliance violations as the features get created by the development or infrastructure teams instead of right before we’re trying to deploy them to production.

Identifying bugs and vulnerabilities early in the process

It’s important for issues that can cause vulnerabilities to be caught as early as possible in the DevOps pipeline.

The reason is that the further into the pipeline you are, the more expensive it is (in both monetary and time requirement terms) to fix issues.

With the traditional DevOps process, code or infrastructure changes are committed and pushed to the next phase. The QA team and automated tools receive those changes and start testing, and everyone crosses their fingers that no major issues are found so that everything can get deployed to production on time in order to meet a deadline.

Except that more often than not, the QA team or automated testing will find problems. Not just cosmetic problems but potentially major security issues that require costly re-writes.

Now, all of a sudden, the issues are sent back to the development or infrastructure teams, and they need to be fixed as soon as possible because they’re holding up a release. Those teams are under massive pressure to deliver a solution. Once they submit a solution, the QA and security tests start over again.

Security becomes a source of pain and anxiety, and when stakeholders ask why a project is delayed, they’re told that it’s because the security team found issues.

Instead, if you shift security left and include it earlier in the process, it becomes integrated into the development process rather than being an add-on further down. Those otherwise costly security issues get discovered much faster and closer to the developer or infrastructure engineer so that they can address those issues before moving on to another task.

Our goal is to give Dev and Ops fast feedback on the work they are producing so that they can be notified right away and can address the issue before moving on to another task and switching contexts.

Making security more manageable

An added benefit of shifting security left is that it helps make security tasks more manageable in a way that can feel completely natural to development teams.

Enabling IT teams to deliver value faster is not a new concept…development has had very similar problems. How has development fixed it? By going agile and by breaking down large projects into the smallest tasks possible.

Security should be handled in the same way. Instead of keeping huge security tests or projects, we need to take those and break them down into really small tests or projects that happen continuously throughout the DevOps pipeline. Being reactive to changes halfway down our deployment pipeline makes this very difficult. Being proactive to changes at the very beginning enables us to be agile.

How do we achieve these benefits?

The way that we achieve these benefits is by getting management buy-in, making security part of everyone’s responsibilities, and by implementing the proper tools and processes.

Let’s talk more about what that means and what it looks like.

Management buy-in

Properly implementing DevSecOps is a fundamental shift in how your organization thinks about, builds, and deploys software and infrastructure. Doing that requires far more than just your peers agreeing to make a change…it requires management buy-in.

Software vulnerabilities being exploited in production results in risks for the organization. The person who judges risks vs. revenue needs to see the value in implementing DevSecOps, or else it all falls flat.

We want to build a mindset that everyone is responsible for security. That sounds cliché, but it’s true, and it is practical, not just conceptual. For good security decisions to be made at speed and scale, they simply can’t be centralized.

If you want to see this properly implemented, you need management to not only back it up but also implement it themselves.

Security is everyone’s responsibility

DevSecOps makes security part of the development process instead of lobbing it over the fence to a security team and forgetting about it.

Instead of hearing, “I don’t usually worry about vulnerabilities in my code or config files because the security team will take care of it” we want to empower the development and infrastructure teams to write more secure code and configurations.

Of course, security is a profession, and we do need highly skilled people in security-focused roles, but we also need everyone else to be responsible for securing their work instead of lobbing it over the fence and saying, “not my problem.”

Again, achieving this requires management buy-in and measuring the right metrics.

Security and DevSecOps training

Getting organizational culture right is critical, but it’s not going to happen overnight. Unless you’re already implementing a lot of these concepts, it’s going to require a fundamental shift. There will be resistance, and it will initially cause frustration because you’re changing people’s traditional ways of thinking and working.

Of course, management buy-in will help alleviate these issues, but so will proper training. Not only does management need to understand these concepts (even if at a high level), but so does everyone else (at a more practical level).

Once everyone understands the DevSecOps approach, they’ll not only understand why it’s important, but they’ll also see how to properly implement it and how it will end up helping them do their jobs more efficiently.

As mentioned before, DevSecOps doesn’t have just one right way of implementing it. With that said, visual learners may find it helpful to visualize the lifecycle.

As we can see, a lot of verifications should happen before we even reach the test or release stages of the lifecycle…or in other words, on the left.

When we’re in the planning phase, which is where all projects start, we should be looking at:

• Providing secure code training
• Creating baselines
• Creating threat models

Then, when developers start writing application code or when infrastructure engineers start configuring cloud resources to support those applications, we should be using:

• Source code versioning
• Code reviews
• Local secrets scanning
• Linting
• Static Application Security Testing (SAST)
• Software Composition Analysis (SCA)

The code then moves on to the build stage, where we should be using:

• Dynamic Application Security Testing (DAST)
• Interactive Application Security Testing (IAST)
• Centralized SCA and SAST (instead of local only)

After that, we have the testing stage which should include:

• Centralized secrets scanning
• Vulnerability assessments

We could even include penetration tests in this stage depending on the size of the organization.

In the release stage, we can run:

• Compliance checks

Then, in the deploy stage, we’ll want to:

• Implement code signing
• Harden endpoints
• Harden applications
• Encrypt everything that needs to be

Once changes are in production, we’re not done. We need to implement:

• Centralized logging
• Process for patching
• Vulnerability disclosure
• Vulnerability assessments
• Penetration Tests
• Bug bounty programs

All of that feeds into monitoring where we should implement:

• Access monitoring
• IPS/IDS and WAF solutions
• SIEM
• Attack surface mapping

All of that helps us learn and improve, which we can then use the next time we start a new project, which takes us all the way back to the planning phase.

Another common way of visualizing and calling this process is CI/CD, or Continuous Integration and Continuous Delivery/Deployment.

Continuous Integration helps us move through the Plan, Code, Build, Test, and Release stages. 

Continuous Delivery helps us move through the Deploy, Operate, and Monitor stages so that a member of the team can then press a button to smoothly deploy all of the changes to production. 

Continuous Deployment is the same as Continuous Delivery, except it aims to completely automate deployment without even requiring manual intervention. Getting to this point requires a serious amount of trust in your pipeline and is a great goal to strive towards.

Getting organizational buy-in and detecting security issues as early in the pipeline as possible sounds great and all, but also much easier said than done. Plus, there’s not just one right way of achieving DevSecOps. Every organization is different in the way that they build and deploy. With that said, there are concepts that can be applied across the board.

We’ve barely scratched the surface as you could see from the graphic above. Each stage has a number of actions, processes, and tools we can (and should) implement depending on our needs.

Now that we understand what DevSecOps aims to achieve, why it’s advantageous, and what it looks like, we need to explore how to properly implement processes and tools to achieve it in our organization. This will be covered in the next article of our DevSecOps series.

Be sure to check back soon for the rest of this series! Better yet, subscribe to our newsletter so that we can notify you when the latest articles get released.

In the meantime, go ahead and share this article with your colleagues so they can start learning about DevSecOps, and learn how a platform like Spacelift can help you and your organization fully manage cloud resources within minutes. Spacelift is a CI/CD platform for infrastructure-as-code that supports tools like Terraform, Pulumi, Kubernetes, and more. For example, it enables policy-as-code, which lets you define policies and rules that govern your infrastructure automatically. You can even invite your security and compliance teams to collaborate on and approve certain workflows and policies for parts that require a more manual approach.

While we will cover concepts like these in the rest of our DevSecOps series, get a head start with Spacelift’s documentation.