AI Integration

Attributes

Goal

The GitLab AI Integration Working Group aims to define, coordinate and ramp up integration of AI capabilities into all product areas

Overview

Corresponding Epic

We want to enable all product teams to be able to use advanced AI capabilities for improving and adding functionality to the product so users can be faster and more productive in their DevSecOps lifecycle. For product teams we want to establish a clear and fast way for going from idea, experiment to production when using AI functionality. Incorporating the services, models and knowhow that the MLOps group has built over time and can provide to the wider team.

The working group will facilitate fast experimentation and prototyping of AI capabilities. We will also advise on what must be considered (and in some cases, get explicit approval on) before moving to production, including legal approval, ethical use of AI, potential necessary changes to terms of service, performance implications, hosting cost implications, infrastructure readiness, security readiness, licensing of 3rd party software/services, appropriate GitLab licensing levels for features, value add in helping users achieve their goals and needs, etc.

More information on the effort and plans can be found in the internal handbook.

Goals

This is a list of topics that we want to discuss:

• Classification of AI possibilities and complexities
• Ideation and understanding how experimentation can be done
• Rapid ML Prototyping
  • API’s and Framework for experimentation
  • Feature Flagged Prototypes
  • Determine how teams without Product Design representation can progress with validation
• Groundwork
  • Base API’s for all teams
  • Infrastructure
  • Documentation
  • Understanding of Jobs and user needs to be affected
• Existing ML features
• Road to production for features
  • Different gates

Exit Criteria

The following criteria should be met for the group to disband:

• Product teams have a clear method to build and integrate AI into GitLab product areas.
• The integration platform should have a product group handling maintanence and feature development
• We have a structured methodology for evaluating new AI models, adding them to the integration platform to allow them to be consumed by product teams.
• We have a roadmap plan to achieve GA for our initial AI experiments.
• Documented process for handling AI feature proposals as part of the prioritization framework
• Move SAFE content from the internal handbook to the public handbook where appropriate and SAFE.
• Develop an evaluation process of user experience between options to make more intelligent decisions on which engineering solution we recommend.

Q2 OKRs

Deliver X experimental, Y beta, and Z GA AI features

Roles and Responsibilities

Engineering Groups

We currently have two core AI Development groups at GitLab: AI Framework group and AI Model Validation group.

AI Model Validation group

The AI Model Validation group helps all product groups to match the right model(s) and AI/ML-based techniques to the user problem they must solve. They do that by evaluating, building, training, and tuning many of the models GitLab uses as well as by proactively sharing AI resources and experience. Today, they also directly build and maintain some user-facing AI features.

• AI Model Validation group develops in-house AI features that run natively within GitLab infrastructure. The group develops these models to meet capability, quality, customizability, privacy, and cost requirements. These components include an inference engine, abstraction layer, and models.
• The custom-built models help in use cases when there is a need to train models on customer-proprietary data (like all merge requests and commits for a customer) and when 3rd party models do meet our needs.
• Their currently released features are code suggestions which is currently in customer beta and suggested reviewer which is GA.
• This group can help evaluate models for functional correctness and model perplexity based on metrics and (often large) benchmark datasets, which is a more statistical evaluation than manual testing. This can help determine the most quality model for a feature’s use-case.
• They work in many languages, including Ruby on Rails, Golang, Python (for machine learning and data science), and Typescript (for the VS Code Plugin). The group comprises of ML Scientistists, MLOps Engineers , ML Infrastructure Engineers, and Fullstack engineers.

You can contact this group via Slack in #g_ai_model_validation. View their issue board here. To see who is engaged on this effort please see here.

AI Framework

AI Framework exposes AI services and the underlying models (third party or native GitLab models) to all product groups.

• The AI Framework group enables the rest of the development department to build AI features through the abstraction layer.
• The abstraction layer supports OpenAI and is being extended to support equivalent Google AI functionality. Other commercial, open-source, and GitLab custom-built models are also being considered.
• This group empowers other groups to evaluate models via manual human testing, through the Experimentation API.
• This group works with Ruby on Rails as they make it easy for the GitLab product to add AI functionality through the gitlab/gitlab-org repo.

You can contact this group via Slack in #g_ai_framework. View their issue board here.

AI Engineering Allocation

Because of the dynamic nature of the AI work and folks to be engaged, we are putting the AI work under an Engineering Allocation. This means that assignments may change rapidly as focus and priorities shift. Current focus is on Code Suggestions adoption.

View page source - Edit this page - please contribute.