CI/CD Build Speed (time-to-result)

Attributes

Overview

We aim to create a working group to establish a repeatable process and framework to measure CI build speed and performance (time-to-result). The objective is to communicate to the market and customers a GitLab point of view, position GitLab CI build performance as the market leader, and provide guidance to customers on optimization considerations so that they can maximize developer efficiency while balancing compute costs for CI builds.

Context:

CI build speed and performance (time-to-result) and CI build cost efficiency are essential competitive vectors especially given the improved maturity of the CI/CD solutions in the market. The brand Q4 FY23 qualitative research study data indicates that “GitLab leads the pack in associations with speed.” However, our internal benchmark testing, (slides, report, of CI build performance on GitLab SaaS had mixed results. Therefore GitLab SaaS customers’ perception of CI build performance could be different than self-managed customers or even GitLab SaaS customers that choose to manage their own CI build environment.

Additionally, to date, competitors, (Github, Harness.io), have generally focused on build time duration when discussing CI build speed and performance. While build time duration is the default performance measure, the working group must evaluate if there are other discrete measurements to include in the framework.

Business Outcomes

1. Establish a repeatable process and framework to measure CI build speed and performance. The process must include the frequency of measurements (quarterly, semi-annual, annual).
2. Review and agreement by legal on how we can communicate CI build speed data externally in blog posts or other marketing materials.
3. Conduct an in-depth analysis of the risks from competitors product features that are focused on improving CI build speed and efficiency.
4. Recommendation to product leadership regarding new investment or prioritization of features to improve GitLab CI competitiveness specific to build speed and performance.
5. Establish a GitLab point of view and voice in the market on the topic of CI/CD build speed and cost efficiency.

Exit Criteria

1. CI build speed benchmark process codified in the handbook.
2. Initial blog post published covering GitLab CI build speed and performance.
3. Sisense charts added as Product PIs for monitoring.

Roles and Responsibilities