Application Security Testing, Vulnerability Research
Vision
Facilitate the Application Security Testing stage achieving lovable maturity and supporting the best possible security assessment at the earliest possible moment.
Customer outcomes we are driving for GitLab
Vulnerability Research sits at the crossroads between the Application Security Testing stage itself, and customers of the stage. We provide enhancements to our products or processes to ensure our products and services are more effective for GitLab’s customers.
We strive to enhance our customer experience with regards to providing correct and accurate results from our services.
Vulnerability Research Team
Priorities
Vulnerability Research is a research & development team. While we do not typically develop or maintain production features, our work directly impacts the product.
Our priorities are:
- Perform security research and develop proofs of concept that strengthen GitLab’s security product offerings, focusing on advancing analyzers. Proofs of concept are split between high-confidence short-term bets that are highly aligned with the product roadmap and low-confidence long-term bets that prove out hypotheses about what our customers will need in the future, as well as new technical approaches that may be novel or unique to GitLab.
- Maintain the advisory database(s) (GitLab Advisory Database and GitLab Advisory Database (Open Source Edition)).
- Carry out CVE Numbering Authority (CNA) duties. GitLab is a CNA.
- Publish papers and write blog posts that are of interest to our GitLab users or those in the security domain.
- Present at security and software engineering conferences to learn and improve our vulnerability research or demonstrate GitLab as a thought leader to the GitLab user base and those in the security domain.
- Write patent applications for unique security technologies.
- Improve the security of open source software.
2024 Q1 top priorities
- Enhance SAST and DAST rules to provide better results to our customers to increase true positives while reducing false positives OKR
- Study false positive trends to identify rules and engines that need optimizing
- Collaborate with development teams to complete the transition to semgrep for multiple analyzers Epic
- Work with development to launch GitLab’s Next Generation SAST Engine OKR
- Publish security blogs and presentations to promote GitLab as a security thought leader OKR
- Write patent applications on new inventions in the security space
Common Links
Mission
Our mission is to advance GitLab’s security offerings toward their long-term vision and to elevate GitLab’s prominence in the security community. We do this by performing security research, working to improve the efficacy of GitLab’s security capabilities, developing proofs of concept, publishing papers, speaking at conferences, and broadly sharing our security expertise and practical experience. You can follow updates to our brown-bag sessions and the GitLab blog (#security and #security-research tags).
How we work
Ideate
- Type of improvement: DevSecOps adoption, Revenue, Security Quality/Efficacy, Cost, Efficiency
- Publication potential: can we present, blog about, or patent this work?
- Evaluation criteria: how will we measure progress and success?
- Reviewers: Secure Product Managers and Engineering Managers
- Tracking: We use the
group::vulnerability research label on gitlab-org/gitlab issues as our SSOT
Prioritize
- Success rate: Balance short-term and long-term bets
- Publication: Potential for using as a source for something public or a patent
- Timing: POC projects should always be aligned with short- or long-term product vision (as appropriate), should be developed in consultation with the product team that will eventually inherit it, and an anticipated timeline for that expected handover should be jointly understood.
Prototype
- Timebox: Short-term bets should be mature enough within three months to decide whether to continue them. The time frame for long-term bets is within six months.
- Customers & prospects: Identify those outside our team to help validate the prototypes (including coordinating with PM and UXR)
Build and Ship
The vulnerability researcher’s top priority is, as needed, to support the engineering team until the feature becomes available in GitLab. A secondary focus is to publish things publicly (blogs, talks, etc.) and file patent applications.
Metrics
Metrics we are working on:
- Reported rule false positive rate - in flight
- Mean time to publish advisory database updates - in discussion
- Mean time to respond to publish new CVEs - in discussion
Static analysis rule update group
The static analysis rule update group focuses on improving the precision and recall of our SAST ruleset; the former by reducing false positives and the latter by reducing false negatives. Work related to this effort is tracked in this Epic and this Issue Board.
We prioritize rule updates and additions based on product management direction, which includes taking into account maximum customer value such as these factors:
- Severity of the rule, because:
- Customer workflows are more likely to be disrupted by high-severity rules.
- False-negative results are more concerning if they fail to report a higher-severity vulnerability.
- Customer escalations about the rule or rules due to inaccurate or missing findings
- Benchmarking results and platform-wide data analyses with a focus on false positive rates
We prioritise our work by applying SAST::Ruleset labels ranging from P1 to P4, with the former being the highest priority and the latter the least.
SAST rules team
SAST rules team communication
We communicate async first, starting with issues and merge requests. We also have a Slack
channel #sast_rule_contractor_team for discussions, #sast_rule_contractor_team_updates for daily virtual standup updates, and a bi-weekly meeting on Tuesdays titled “GitLab SAST Rules bi-weekly sync”.
We use the rule enhancement processes to guide our workflow (how to enhance a rule, how to update a rule, rule reviwer checklist, etc).
Workflow label usage
The team uses these workflow labels for issues:
- Status
Open - The issue is not vetted nor prioritized. Vulnerability Research (VR) with collaboration with PM as needed will prioritize then add the label workflow::ready for development
workflow::ready for development - The issue is ready for an engineer to start work. These will always have a SAST::Ruleset priority label. Developers choose from this list based on the priority of the language per this epic, their skills in those languages, the priority of the rule based on the SAST::Ruleset priority label, and estimated level of effort (lower effort rules first so customers see results earlier).workflow::in dev - A developer is working on it and has assigned the issue to themselves and added this label.workflow::blocked - Work cannot continue on this due to an outside dependency. This can be set by anyone collaborating on the issue, generally the assignee. When using this comment as to why the issue is blocked or link the blocking issue.workflow::in review - The MRs associated with the issue are in review by the picked and assigned reviewer(s). This label is applied by assignees to the rule’s issue once the development is complete.workflow::complete - The MRs associated with the issue are all merged. The issue assignee sets this label.- Status:
Closed - The MRs associated with the issue are incorporated into a rules release that is available to customers. The issue assignee sets this state.
This handbook page is intended to document CNA processes that the
Vulnerability Research team uses in contributing to
GitLab’s role as a CNA.
CVE Requests
CWEs, CVSS Scores, and Descriptions
-
Start with identifying an accurate CWE for the vulnerability
-
Review the CVSS score that the submitter provided
- If the CVSS score is largely out-of-line with what you would expect
based on the CWE and the description, confirm with the submitter that the
score makes sense
- If clear reasons exist for the unexpected CVSS metrics, add a note in the
description to this effect. For example, "… Overall impact is limited
due to the user only being able to affect their own account"
- Note The CVSS score should make sense from an outside perspective
when only having access to the CVE description and CWE
GPG Key
The email cve@gitlab.com has
a GPG key that the
Vulnerability Research team uses during CNA procesess.
Dinesh Bolkensteyn
Isaac Dawson
Michael Henriksen