GitLab Delivery:Framework

Summary

Framework is a team within the GitLab Delivery Stage. This team takes ownership of the GitLab Environment Toolkit (GET) as instance lifecycle tool, Reference Architectures, tooling and frameworks related to upgrade path testing, and will ensure installation and upgrade paths are fully tested and reliable. We follow the same processes as listed on the GitLab Delivery Stage, unless otherwise stated on this page.

Mission

The Framework team intends to make GitLab easy to operate at any scale. GitLab is critical to our end users (developers) and a necessity for organizations most important business operations and workflows. The Framework team functions with autonomy to drive good quality within our Reference Architectures and GET above all else.

Some optimal success criteria include

  1. A decrease in the cost of support per customer
  2. Lower time to value for new installations
  3. More instances on the recent version of GitLab
  4. Fewer upgrade-related incidents
  5. Instances on supported architecture

Seeing that many of these metrics are owned by multiple teams, the Framework Team will contribute here but does not have full agency to drive.

What does it mean to Operate GitLab?

GitLab is a complex application. Organizations typically have a dedicated team to manage GitLab internally. This operation includes day-to-day maintenance, upgrades, backup and restore, and downtime. This team’s goal is to make this operation easy for all organizations, including our own deployments.

It is not solely the responsibility of the Framework team to make GitLab easy to operate, but to provide guidance to the wider GitLab organization to develop in a way that is conducive to a good user experience for admins. Managing GitLab components in a tested and supported architecture significantly lowers the operational aptitude required.

Delivery of Mission

  1. Easy enablement materials to consume reference architectures for account teams and operators
  2. Collaborate on structured guidance for operational excellence in feature development
  3. Verify upgrade paths
  4. Stewards of a good experience for managing GitLab with GET and the Reference Architectures

Tools and Responsibilities

The Framework team owns and leads our GitLab Environment Toolkit and Reference Architectures initiatives, which are key programs with the aim assist and enable customers to deploy and maintain their GitLab environments. This includes external users as well as easing the operational requirements for our own SaaS offerings.

GitLab Environment Toolkit (GET)

GET is a set of opinionated Terraform and Ansible scripts to assist with deploying scaled self-managed GitLab environments following the Reference Architectures. A good experience with GET means users will more quickly have an environment (lower TTV) and long-term success (decreased need for support).

Reference Architectures

The reference architectures will provide the best user experience with the intent to decrease the amount of required support from GitLab. The Framework team will continue to ensure the RAs meet the needs of customers by improving architecture designs over time. The Framework team will also provide guidance for internal teams to review and add components as needed.

Upgrade Path Validation

Upgrading GitLab is one of the most challenging aspects of its operation and often requires dedicated attention to effectively plan the upgrade process. The Framework team continuously validates the upgrade paths between releases to ensure that only necessary, planned stops are introduced, thereby instilling confidence in the upgrade process for our Self-Managed and Dedicated customers.

Other Responsibilities

  1. Architecture performance and non-functional testing
  2. Enablement of GitLab account teams to provide appropriate recommendations
  3. Self-serve experience for scaling GitLab

Team Members

Name Role
Lekan SogunleLekan Sogunle Senior Backend Engineer

Product Manager: Martin Brümmer

Roadmap

The team’s primary roadmap is tracked within the Roadmap epic. Our roadmap is aligned with GitLab Delivery’s overall strategic objectives and is reviewed and updated as needed.

Working with Us

To engage with the Framework team, please refer to the following.

Please kindly note that as an engineering team our resources are finite. Direct asks to join customer calls or projects are not within the remit of this team and should instead be directed accordingly to Product Management, Support or Professional Services accordingly.

Slack Channels

The following list includes Slack channels relevant to projects within the team’s scope:

  • #g_software_delivery-framework - This is the primary Slack channel for discussions related to the GitLab Delivery Framework team.
  • #gitlab_environment_toolkit - For discussions, questions, and requests specific to the GitLab Environment Toolkit (GET).
  • #reference-architectures - For discussions, questions, and requests related to Reference Architectures.
  • Our slack group handle is @software-delivery-framework-team.

Reference Architecture Review requests

For any requests relating to customer environments, either proposed or existing, they must be raised in the Reference Architectures project in the following circumstances only with the given templates:

  • environment-review-request - Sanity checks of new environment designs, not already covered in the Reference Architecture docs.
  • request-for-help - Help requests from the Support Team for assistance when an issue is suspected to be due to the environmental design.

Requests should be opened two or more business days before action is needed to ensure the team has time to prepare and we kindly ask for this process to be followed for tracking and capacity reasons.

Triage DRI

The Triage DRI is a role established by the team to handle initial triage of questions and requests coming to the team. The Triage DRI process is intended to distribute knowledge and responsibilities across the team, and to prevent the same people from always being responsible for certain areas of the team’s work.

Responsibilities

  1. Triaging and monitoring pipelines (e.g., Upgrade path pipelines).
  2. Monitoring and responding to requests in GET/RA Slack channels.
  3. Initial issue triage for GET, and RA.
  4. Maintenance of PG Dump Generator and related jobs if breaking master.
  5. Staging Ref maintenance.

Cadence

The Triage DRI role will be on a weekly rotation cadence. Regarding response time, we’ve set an aspirational goal of about one working day. It’s important to note that this isn’t a strict Service Level Agreement (SLA). We’re viewing it more as a general guideline to aim for. We understand that sometimes things might take longer, and that’s okay. The main goal is to have a reasonable timeframe for addressing inquiries and issues that come our way.

It must be emphasized that the Triage DRI is not an escalation resource like Support, but rather an advisory one. The Triage DRI is there to provide advice and initial triage, not to be a replacement for Support or other teams.

Project Management

Framework Issue Tracker. GitLab Environment Toolkit (GET) issues - Primary tracking system for the GitLab Environment Toolkit. This board encompasses requests, enhancements, technical debts items and customer reported issues.

GitLab Upgrades Test Coverage - This epic encompasses issues related to testing upgrade paths.

Reference Architecture issues - This is where we track work related to Reference Architectures and conduct reviews for customer environments upon request.

Epics

This top-level epic monitors the different initiatives and sub-epics within the team and their respective statuses. It is populated automatically through epic automation. Every sub-epic must

  1. Epics must be created with the epic template linked below.
  2. Have the top-level epic as the parent epic.
  3. Have a completion date and a DRI for providing a weekly status update.
  4. Have a relevant workflow:: or workflow-infra:: label.
  5. Epic DRIs will be pinged on Tuesdays to leave a status update for the week in the epic.
  6. These updates are collected into the team’s top-level epic which feeds into the Grand Review.

We follow Platforms Project Management practices as outlined here.

Labels

The primary team label is team::GitLab Delivery Framework. This label should be applied to issues and epics throughout the organization that require the team’s attention. The team tracks work using the workflow and workflow-infra labels across issues, epics, and MRs. Team members will ensure that their work items are updated with the correct workflow labels.

gitlab-org issues gitlab-com/gl-infra issues
~“workflow::refinement” ~“workflow-infra::Triage”
~“workflow::ready for development” ~“workflow-infra::Ready”
~“workflow::in dev” ~“workflow-infra::In Progress”
~“workflow::in review” ~“workflow-infra::Under Review”
~“workflow::blocked” ~“workflow-infra::Blocked”
~“workflow::verification” ~“workflow-infra::Verify”
~“workflow::complete” ~“workflow-infra::Done”

Epic Template

Epic Template
### DRI :levitate:
- TBC

### Participants :busts_in_silhouette:


### Problem to solve :thinking:

### Why :results-for-customers:
<!-- Explain the importance of this epic and its contributions to delivering results for customer. -->

### Future Maintenance :construction:
<!-- If this involves building a new piece of tooling, pipelines, or artifacts, clearly identify and align on future maintenance responsibilities. -->

### Documentation :book:

* Publicly Accessible Documentation:

### Links / references :books:

*

---

<!-- STATUS NOTE START -->

<!-- STATUS NOTE END -->

/label ~"team::GitLab Delivery Framework" ~"workflow-infra::Triage"

Team Impact

The following are records of the annual achievements of the Framework team.

  1. Team Impact Overview for 2024