Vulnerability Management

Vulnerability Management is the continual process of identifying, prioritizing, mitigating and remediating vulnerabilities. At GitLab we identify vulnerabilities in a number of different ways depending on the component being analyzed. This process and associated tooling is owned by the Vulnerability Management team.

This page primarily outlines our vulnerability management policies and procedures at GitLab. The policies and procedures used to manage vulnerabilities at GitLab are collectively referred to as the Vulnerability Management Standard.

Quick reference

What does Vulnerability Management cover at GitLab?

Vulnerability Management covers all systems which store, process or transmit GitLab production and/or GitLab customer data, as well as GitLab-managed software and software dependencies, based on the GitLab critical system tiering methodology. Specifically:

  • GitLab-managed infrastructure
  • GitLab software, packages, images & dependencies
  • Detected Later (HackerOne reports, Community or Team Member reported bugs, PenTest findings)

Approach & Strategy

Vulnerability Management at GitLab focuses primarily on gathering data about vulnerabilities, vulnerability detection and attack surface. We do this by building tooling to gather, normalize and correlate various sources of data into actionable automated workflows which either directly address or mitigate findings. If this cannot be done automatically, we aim to make the output of this process easily available and understandable to the GitLab DRIs responsible for mitigating or fixing a finding. Our goal is to feed this tooling back into GitLab itself, and where it makes sense, to build and use GitLab features to support our own and our customer’s workflows wherever possible.

Vulnerability Management Standard

  • Where possible, GitLab should be used as the source of truth for vulnerability detection and management. GitLab issues are currently used to track vulnerability management activities in addition to engineering work required to remediate vulnerabilities.
  • Application (WebApp and container) vulnerability scanning must occur on a minimum of a monthly basis, and use scanners and scanner configuration tuned for the code being scanned. This includes enabling appropriate dependency scanning, container scanning, secrets scanning, and SAST scan jobs. See the Secure your application documentation for all available scanners. Where possible, standard supported GitLab scanners should be used in CI based on supported templates or components.
  • Container images shipped by GitLab should be scanned at a minimum with GitLab container scanning before being shipped.
  • Infrastructure (operating systems and databases) vulnerability scanning must occur on a minimum of a monthly basis.
  • Authenticated/credentialed scans from a privileged network domain, in addition to external scanning of infrastructure, should be performed wherever possible.
  • Vulnerability scanners must be up to date (the latest operable version), and hardened to resist unauthorized use or modification.
  • Third party penetration testing must be performed annually.
  • Even if mitigated or not exploitable, vulnerable code (including dependencies) must be removed or updated to non-vulnerable versions to resolve vulnerability findings, so we are not shipping vulnerable code. For more information on why we do this, please see Why should we fix vulnerabilities?
  • Vulnerabilities must be fixed within SLA timeframes, unless an exception exists. Where vulnerabilities are not fixed within SLA, the responsible group should work to understand which factors contributed to missing remediation SLA timeframes as part of regular group development retrospectives.

Service Level Agreements (SLAs) / Service Level Objectives (SLOs)

For information on vulnerability management SLAs / SLOs, see the Vulnerability Management SLAs handbook page.

Roles & Responsibilities

Role Responsibility Notes
Vulnerability Management Responsible for implementing and maintaining Vulnerability Management standards. Develop and maintain (VulnMapper). Evaluate severity and priority, analyze the actual impacts, and make risk adjustment for the vulnerabilities which VulnMapper is unable to process automatically. Iterate on data gathering, normalization and correlation automation and work to provide actionable insights on mitigation and remediation to GitLab team members.
Security Compliance Team Responsible for oversight of supporting procedures as part of ongoing continuous control monitoring for adherence with compliance baselines. Responsible for approving Deviation Requests (or getting approval from Agency) and closing DR issues once all linked vulnerability issues have been closed.
Application Security Team Collaborate with Development & Engineering in the triage of vulnerabilities and provide stable counterparts TODO.
Development & Engineering Responsible for mitigating and remediating vulnerabilities in GitLab-managed software within SLA
Reliability Responsible for mitigating and remediating vulnerabilities in GitLab-managed infrastructure within SLA
Product Security Engineering Team Develop and maintain the GitLab Security Bot, a.k.a. appsec-escalator, which drives SLA and remediation reminders AppSec collaborates with Product Security Engineering and Vulnerability Management in maintaining this bot.
Infrastructure Security Responsible for maintaining cloud configuration and resolving vulnerabilities or misconfiguration in cloud policy and other security configuration
Security Leadership (Code Owners) Responsible for approving changes to this standard

Additional Notes for Vulnerability Triage

  1. VulnMapper creates vulnerability tracking issues and if not available already, adds appropriate severity, priority and other necessary labels. Vulnmapper is currently opt-in for issue creation, if issues are not created for your group, please let Vulnerability Management know.
  2. For the vulnerabilities which VulnMapper can process automatically, Vulnerability Management updates vulnerability tracking issues with the necessary information such as CVSS score, severity and priority, package name and version etc. Current capabilities are documented on the automation handbook page.
  3. For supported OS bases, VulnMapper also adds labels indicating fix availability to further reduce triage time, based on vendor advisory and package information. Supported OSes for this functionality is limited, and is detailed in the automation handbook page.
  4. Where fixes are not available or differences between NVD and vendor severity exists, VulnMapper reports Deviation Request issues for vendor dependencies and vendor risk adjustments automatically.
  5. Vulnerability Management assigns vulnerability issues to a development group based on available information. VulnMapper also is configured with a map of ownership information for GitLab development groups, and will appropriately label issues for different groups with the appropriate triage group labels, when updating or creating issues.
    1. Where possible, Vulnerability Management, via our automation, where available, will make recommendations on how to address vulnerability detections, either via an MR or the description in vulnerability tracking issues.
    2. Vulnerability triage is a shared responsibility although Vulnerability Management is the DRI. AppSec, InfraSec and respective feature domain development groups are responsible for providing evidence and/or analysis of GitLab-specific impact during vulnerability triage upon Vulnerability Management’s request. Vulnerability tracking issues should be assigned to the appropriate AppSec stable counterpart or development group for further analysis and after this, for remediation, the responsible development or infrastructure group becomes the owner and DRI, as they will have the context and responsibility over impacted vulnerable assets.
  6. Vulnerability is now ready for remediation by the responsible group

Contact

If you have any questions or concerns related to vulnerability management please contact Vulnerability Management and Engineering in the #g_security_vulnmgmt or the #security channels on Slack, you can also open an issue in the Vulnerability Management issue tracker. All work being done to improve this process is also tracked in the issue tracker, and we’d love your feedback.

Exceptions

If a you are mitigating or addressing a vulnerability, and you do not believe the GitLab Vulnerability Management standards can be met, please raise an exception request. For details on exception requests, handling, including raising a request for an SLA exception, see the SLA Exception handbook page.

References


Closing Vulnerability Tracking Issues
At GitLab, we use GitLab to track the detection and remediation of vulnerability findings. In the …
Development Labels
Labels used in Vulnerability Management Engineering for scoring and sizing automation development bugs and feature issues
Encryption Policy
Purpose This policy is intended to outline the encryption controls and requirements at GitLab. Scope …
Incident Response Guidance
This guidance will provide all in scope individuals the information they need to help GitLab ensure incidents are reported, investigated and handled.
Infrastructure Vulnerability Management Procedure
This procedure applies to vulnerabilities identified in GitLab Production infrastructure and ensures …
SLA exceptions
SLA exception take on two different forms at GitLab, with two slightly different processes. This …
Vulnerability Lifecycle
Tracking Issue Lifecycle The typical lifecycle of a vulnerability detection at GitLab, at a high …
Vulnerability Management - Standard Issue Labels
A standard set of vulnerability management labels are used across GitLab to assist team members in …
Vulnerability Management Automation
Vulnerability management maintains automation tooling which intends to automate as much of this …
Vulnerability Management Code Review and Development Standard
Purpose This standard documents requirements for all engineering & development work which …
Vulnerability Management Definition: What Does Fixed Mean?
Overview At GitLab, when a vulnerability finding is detected, it is detected against one or more …
Vulnerability Management Team
This handbook page describes how the Vulnerability Management team works on a day to day, quarter to …
Vulnerability Management Team Runbooks
This handbook page lists runbooks created by the Vulnerability Management Team Runbooks Fixing …
Vulnerability Resolution SLAs
Vulnerability Management SLAs and Labels The following timelines or service level agreements (SLAs) …
What is a vulnerability?
What is a vulnerability? A vulnerability describes a flaw in either code or configuration with …
Why should we fix vulnerabilities?
Overview Sometimes we encounter vulnerability findings in GitLab or GitLab systems, and through …