CI/CD Build Speed
Status | Authors | Coach | DRIs | Owning Stage | Created |
---|---|---|---|---|---|
ongoing |
grzesiek
|
grzesiek
|
gabrielengel_gl
|
2024-01-12 |
Summary
GitLab CI is a Continuous Integration platform which is widely used to run a variety of jobs, builds, pipelines. It was integrated into GitLab in September 2015 and has become one of the most beloved CI/CD solutions.
With years we’ve added a lot of new features and code to the GitLab CI platform. In order to retain the “one of the most beloved solutions” status, we also need keep attention to making it fast, reliable and secure. This design doc is describing the path towards the former: making GitLab CI fast by improving CI build speed.
Goals
- Establish a CI Speed Benchmark, used to compare GitLab CI to other platforms.
- Build CI Benchmark Framework to measure the GitLab CI speed over the long term.
- Describe next steps for improving GitLab CI Build Speed.
Proposal
CI Speed Benchmark
First, we plan to build a CI Speed Benchmark solution, that will allow us to run specific scenarios on various CI/CD platform and ingest results into our data warehouse.
This will make it possible to define a baseline of the CI Build Speed for many different scenarios and track the progress we, and other providers, are making over time.
The core part of this goal is to define a set of scenarios that will allow us to build a proxy metrics for build speed. For example, we could run following scenarios:
- Time to first byte of build log for
echo "Hello World"
build. - Time to result to perform a CPU-intensive cryptographic operation.
- Time to result to perform a memory-intensive for a given amount of bytes.
- Time to result to build a Linux kernel.
The scenarios should be idempotent and deterministic.
In the first iteration, we will only focus on the total job execution time, and not go into detail e.g. comparing specific startup times.
CI Benchmark Framework
Once we define scenarios that we want to implement, we should build a CI Benchmark Framework. The framework will be used to run scenarios in a Continuous Integration environment, and to send the results back to our data warehouse, for analysis and comparison.
The main principles behind design choices for the framework, are:
- Make it CI-platform agnostic. Can run on any Continuous Integration platform.
- Do not depend on any specific technology that might not be available on some platforms.
- Easy installation setup, not requiring many dependencies. Zero-dependency would be ideal.
- Send results back to GitLab through an HTTP request, unless there is a better way.
Improve CI Build Speed
Once we can measure CI Build Speed, improving it can be possible. We will define the next steps for improving the speed once we have initial results.
e47101dc
)