Vulnerability Management

At GitLab, we use GitLab to track the detection and remediation of vulnerability findings. In the case of remediation work, a vulnerability tracking issue (which is a regular GitLab issue) is used to track work required to remediate a specific vulnerability finding, typically against one or more assets where the vulnerability was detected.

In order to close a vulnerability tracking issue, one of the following criteria must be met:

1. The finding has been remediated and validated as remediated, and made available to intended users
2. The finding was a false positive detection, and required work has been performed to prevent re-detection
3. A finding was in a third-party dependency from a vendor, and the vendor has indicated they will not fix the vulnerability due to lack of impact, and an SLA exception or Deviation Request has been raised

In other situations, there is typically more work which is required before the issue can be closed, to ensure the detection is not erroneously closed and the finding forgotten about, and issue closure is not appropriate as a result.

Purpose

This policy is intended to outline the encryption controls and requirements at GitLab.

Scope

This policy is applicable to the production environment and any end user devices that store such data. This also includes the GitLab Dedicated single-tenant SaaS offering.

Roles & Responsibilities

Policy

Encryption

Customer data is encrypted at rest. (SC-28)

This procedure applies to vulnerabilities identified in GitLab Production infrastructure and ensures implementation of the Vulnerability Management Standard. This procedure is designed to provide insight into our environments, promote healthy patch management among other preventative best-practices, and remediate risk; all with the end goal to better secure our environments and our product.

Scope

Security and Infrastructure partnered to come up with a scope that would make sure all of our critical environments and systems were covered during deployment. The following environments are currently in-scope for GitLab.com and GitLab Dedicated production:

SLA exception take on two different forms at GitLab, with two slightly different processes. This difference in procedure only takes effect if the asset (image, server, package) where a vulnerability finding has been detected is shipped into and used within the FedRAMP boundary or environment. Typically, at a high level, findings in FIPS compliant images will be in-scope for FedRAMP, and should be treated as such for requesting SLA exceptions.

Tracking Issue Lifecycle

The typical lifecycle of a vulnerability detection at GitLab, at a high level, follows the following steps:

1. Vulnerability detected by scanner or otherwise reported
2. A GitLab issue is created to track the vulnerability
3. The vulnerability is remediated within SLA
4. The issue is closed

Depending on the environment and what is being scanned, the tracking issue will be handled by different GitLab team members. Issues may also be enriched by different automated tools, such as (VulnMapper, which assist with streamlining the detection and remediation process. When an issue cannot be remediated within SLA, additional procedures exist for adjusting or exempting vulnerabilities from SLA. See the section in this page on Risk Acceptance & SLA Exception for detail.

A standard set of vulnerability management labels are used across GitLab to assist team members in handling security vulnerabilities in GitLab-maintained projects. These labels are applied automatically by automation managed by the Vulnerability Management team. Automated labeling happens on a daily cadence. Several data sources are used to enrich existing security tracking issues, and are labels are designed to increase the amount of information and context available to team members about vulnerabilities present in the projects they maintain. The primary goal is to reduce the amount of manual research and triage work required, and help to apply consistent and effective management of vulnerabilities across GitLab.

Vulnerability management maintains automation tooling which intends to automate as much of this standard and the related procedures as possible. We’re always iterating on ways we can save time for all stakeholders whilst keeping GitLab and our users safe.

VulnMapper

VulnMapper is the main automation tool used by Vulnerability Management. It is a closed-source tool with a goal of automating security processes and building functionality into GitLab as part of Product Security’s dogfooding goals.

Overview

At GitLab, when a vulnerability finding is detected, it is detected against one or more assets (such as servers, container images or packages). When remediating or “fixing” a vulnerability detection, it is important that a full remediation of the detected finding is made and validated as no longer being detected or exploitable prior to considering a vulnerability finding fixed.

This handbook page contains several scenarios for different types of detections, and sets out the “definition of done” and what constitutes a fixed vulnerability.

This handbook page describes how the Vulnerability Management team works on a day to day, quarter to quarter basis. We recognize this process is iterative and requires regular introspection to ensure we are always improving and not stagnating in our approach.

Planning

Vulnerability Management follows GitLab product milestones

Prior to a new milestone beginning, planning is performed to define the expected outcome of the next milestone, and the work required to accomplish it. We review our ability to deliver this work within the milestone and commit/adjust to issues to fit. Additionally, any work which is not scoped or not sized is qualified and labelled.

Vulnerability Management SLAs and Labels

The following timelines or service level agreements (SLAs) are based on many factors - such as regulatory compliance, customer SLOs & SLAs, vulnerability impact, scope, prevalence in GitLab environments, impact if exploited, and defining reasonable turn-around times for mitigation and remediation to protect GitLab and our customers. All of these factors will be considered when mapping the priority to GitLab’s priority labels. All components in scope of Vulnerability Management are subject to the same SLAs. The SLAs are as follows:

What is a vulnerability?

A vulnerability describes a flaw in either code or configuration with potential or demonstrated security impact. For a flaw to be considered a vulnerability, rather than just a bug, it must have security impact, potential or demonstrated. Security impact is classified by the potential to cause software to behave in a way which would be considered unexpected by a typical user of the software or system, negatively impacting the availability or integrity of the software, or the confidentiality of information held or processed by the system. A vulnerability does not need to be demonstrated to be exploitable to be considered a vulnerability, as many vulnerabilities are exploited in very creative and unexpected ways, often in combination with other vulnerabilities to form an attack chain.

Overview

Sometimes we encounter vulnerability findings in GitLab or GitLab systems, and through various mitigating factors or a lack of clear vectors for exploitation, it may not seem important to resolve those vulnerabilities, when viewed in isolation. The handbook page details potential reasons team members might not have considered for resolving vulnerability detections in a number of scenarios.

There are several important reasons for fixing vulnerabilities:

1. To remediate direct threat of vulnerabilities being exploited in GitLab or GitLab systems
2. To improve the security of our software and systems, to prevent negative impact to the availability and confidentiality of GitLab user data, using exploits or combinations of exploits we may not yet be aware are possible
3. To demonstrate our commitment to a mature and secure software delivery lifecycle (SDLC). This helps to demonstrate to GitLab users that they can trust our software and systems, by ensuring scans of our systems, images, and packages are as clean as possible as a measure of the health of our procedures

Mitigating factors or a lack of direct exploitation scenarios should not mean we don’t need to fix a vulnerability detection. If you believe a finding to be mitigated or not exploitable, this can be used to justify an exception to regular SLAs, however this should still be addressed and a fix shipped per the above reasons. If you believe a detection is made in error, which is typically referred to as a false positive in a security scan, you can raise an SLA exemption documenting to error in detection. Ideally, we would then feed information about the detection back to development groups (for GitLab security scanning product features) and to third party vendors or projects for all other scanners, to get the false positive resolved at the source.