CI/CD Security: Securing Your CI/CD Pipeline

The CI/CD (continuous integration/continuous delivery or deployment) pipeline is the foundation on which all DevOps processes are built. As such, securing CI/CD pipelines at every stage, and across every tool and environment that comprise the pipeline, should be a priority for any organization that embraces DevOps.

Keep reading for a primer on why CI/CD security is important and best practices for enforcing security across the pipeline.

What Is CI/CD Security?

CI/CD security is a multi-stage process that seeks to identify and mitigate security risks at every stage of the CI/CD pipeline.

The specifics of CI/CD security will vary from one team to another, based on the unique characteristics of each team’s CI/CD operations. Although all CI/CD pipelines include at least a few core stages – source code management, building, testing, and deployment – some also have feature branches that add complexity – and security risks – to the pipeline. Likewise, CI/CD pipelines that target multiple production environments (like Windows and Linux servers) are more complex and involve more security risks than those that push applications into only one type of environment.

The way you approach CI/CD security, then, depends on exactly what your CI/CD processes look like and which tools you use to power them. Nonetheless, the core focus when securing a CI/CD pipeline should be to ensure that no matter which path code takes as it moves down the pipeline, you mitigate all potential risks at all stages.

CI/CD Security Threats

In addition to understanding how your pipeline works, you must also recognize the various types of security threats that could impact CI/CD operations and take steps to defend against each one.

While it’s impossible to identify every type of potential CI/CD security risk before it strikes, common examples include:

• Insecure code that is imported into a CI/CD pipeline from a third-party source, such as an open source project.
• Unauthorized access to source code repositories or build tools, which could allow attackers to inject malicious code into an application.
• A breach of a dev/test environment where an attacker disables an important security test, thereby preventing the team from detecting a security breach at the testing stage of the pipeline.
• Insecure secrets management within the pipeline due to practices such as hardcoding passwords into IaC templates. Attackers who gain access to this information could use it to execute a breach.

How to Secure a CI/CD Pipeline

Because CI/CD threats impact varying stages of the pipeline and different layers of the software stack, CI/CD pipeline security requires a multi-pronged approach. You need to deploy tools and processes that can detect a variety of different threats.

The following are key categories of tools and best practices for helping to detect and manage CI/CD security risks.

Source Composition Analysis

Source composition analysis, or SCA, is the scanning of source code to identify vulnerabilities that originate from third-party modules or libraries, such as those imported from an open source project. SCA tools can scan your source repositories at the start of the CI/CD pipeline and alert you to dependencies that, if left unaddressed, may result in security vulnerabilities within your production application.

Static Application Security Testing

Static application security testing, or SAST, complements SCA by assessing source code that you write yourself for potential vulnerabilities. In other words, whereas SCA can identify vulnerabilities in third-party code based on databases of known vulnerabilities, SAST performs original analysis of custom code to detect potential security issues, like improper input validation.

Thus, by performing SAST at the start of your CI/CD pipeline in addition to SCA, you gain a second layer of defense against risks in source code.

CI/CD Access Controls

Using access controls to manage who can access which tools and resources within the CI/CD pipeline is essential for preventing attacks wherein threat actors inject malicious code into the development environment – which is the attack vector used in the infamous SolarWinds breach that was announced in 2020.

To manage risks like this, each tool within the CI/CD pipeline should be secured with passwords, access keys, or other access controls, and access should be granted on a granular basis only to team members who specifically need it. Although it can be tempting to give all developers and engineers access to all CI/CD resources, this is not a best practice. It deviates from the principle of least privilege and maximizes the chances that a breached account will give attackers the keys to your team’s entire CI/CD kingdom.

Secrets Management and CI/CD

Secrets like passwords and access keys are often required at multiple stages of the CI/CD process to integrate or build code, deploy applications in testing or production, and so on.

Typically, the easiest way to share these secrets is to hardcode them into CI/CD configuration files or the IaC templates that your team uses to provision application environments. But here again, the easy route is not the secure route. Hardcoded secrets can be easily accessed by anyone who is able to view configuration files or IaC templates, which creates a significant security risk.

A better practice is to use a secure secrets manager to store sensitive data and share it on an as-needed basis during CI/CD operations.

Registry Scanning

If your CI/CD pipeline deploys applications using containers, you can add another layer of protection by performing automated scanning of container images as they enter registries. Registry scanning checks containers for vulnerabilities, allowing your team to catch malicious code that may have slipped by the security tests that you performed during earlier stages of the CI/CD pipeline.

Registry scanning is especially important given that some vulnerabilities may be introduced not at the source code stage of the pipeline, but as part of dependencies that are later incorporated into a container. As a result, SCA and SAST tools, which scan only source code, won’t catch these vulnerabilities. Registry scanning will, in most cases.

Runtime Security

Runtime security, which is the detection of active threats in live application environments, helps you manage threats during the final major stage of the CI/CD pipeline: deployment into production. You can also apply runtime security to dev/test environments as part of a “shift-left” security strategy.

Runtime security relies on a variety of tools and data sources, such as application logs, network metrics, and audit logs. Your approach to runtime security will vary depending on what your runtime environment looks like – whether it uses containers, which orchestrator you use, and so on – but the core goal should be to detect live threats before they escalate.

An insecure CI/CD pipeline will inevitably lead to an insecure application. By managing security risks at all stages of the pipeline via practices like access control, secrets management, and registry scanning, you can minimize the risk that insecure code will flow down your pipeline.