MLOps Working Group

Attributes

Goals

This Working Group has the following goals:

1. Begin defining and refine the results driven business value stream of MLOps at GitLab.
2. Formalize the processes related to provenance, storage and access of GitLab.com production data for the purpose of model training.
3. Share knowledge and determine best practices for hyper-parameter tuning, retraining, versioning, and deploying new ML models
4. Determine best practices for benchmarking models built by different frameworks/libraries for different use cases in terms of accuracy and performance, and how to do it in a continuous basis.
5. Determine how to distribute machine learning models on self-managed instances
6. Define a security/legal process for security-related ML models and data pre-processing

Definitions

What is MLOps?

As per Wikipedia, MLOps or ML Ops is a set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently.

Read more about the topic area from the links below:

• MLOps Wikipedia
• MLOps: Continuous delivery and automation pipelines in machine learning
• Machine Learning Operations
• MLOps Slack Group

Related GitLab Documentation

• MLOps Single-Engineer Group
• MLOps Primer
• MLOps Exploration
• Product Stage Direction - ModelOps
• Draft Group Direction - MLOps
• Group Direction - AI Model Validation
• AI Model Validation Group

Related GitLab projects

• Spamcheck
  • Architecture Diagram
• Tanuki-Stan
• UnReview
  • Architecture Diagram

Related ML Slack channels

• #g_applied_ml
• #g_machine-learning
• #feed_tanuki-stan
• #security-ml

Exit Criteria

The charter of this working group is to bridge the gap between different teams that are building ML products at GitLab by discussing overlapping architectural concerns:

1. Creation of a mission statement that the MLOps Working Group operates under, added to the handbook.
2. Create a handbook page discussing a deep analysis of the tooling for ML out there and a proof-of-concept framework using an ideal pathway that GitLab teams can refer to and easily extend when kicking off ML-related projects.
3. Creation of helpers/libraries (presumably in Python) that can be shared across teams and used for the purposes of similar tasks e.g. data access and storage, data pre-processing.

Outcome

1. The Working Group has brought together various teams that are working on different ways to leverage Machine Learning within GitLab the product and in their day-to-day work. It has given these teams a platform to discuss their mission and an opportunity to share knowledge on projects they are currently undertaking and what they are looking to achieve in the near future.
2. We will move forward with the creation of a monthly MLOps meetup that focuses on sharing ideas and showcasing work done by different teams, in a less formalized structure, to enhance awareness across the company.

Roles and Responsibilities

View page source - Edit this page - please contribute.