AI Framework Group

Vision

The AI Framework group is focused on how to support other product groups at GitLab with the AI Abstraction Layer, and GitLab AI feature development.

👌 Team OKRs

If you’re interested in the team’s Objectives and Key Results (OKRs), you can find them on GitLab.

🚀 Team Members

Engineering Manager & Engineers

Name	Role
David O'Regan	Engineering Manager, AI Framework
Alexander Chueshev	Staff Machine Learning Engineer, AI-powered:AI Framework
Alejandro Rodríguez	Senior Backend Engineer, AI Powered:AI Framework
Andras Herczeg	Senior Machine Learning Engineer, AI-powered:AI Model Validation
Hongtao Yang	Senior Machine Learning Engineer, AI-powered:AI Model Validation
Lindsey Shelton	Frontend Engineer, AI-powered:AI Framework
Małgorzata Ksionek	Staff Backend Engineer, AI Powered:AI Framework
Michael Thomas	Backend Engineer, AI-powered:AI Framework
Nathan Weinshenker	Backend Engineer, AI Powered:AI Framework
Stephan Rayner	Senior ML Engineer, AI-powered:AI Model Validation
Tan Le	Staff Machine Learning Engineer, AI-powered:AI Model Validation

Product, Design & Quality

Name	Role
Katie Macoy	Senior Product Designer, AI-powered:AI Framework
Senior Product Designer	Senior Product Designer, AI-powered:AI Framework
Ottilia Westerlund	Security Engineer, Fulfillment (Fulfillment Platform, Subscription Management), Security Risk Management (Security Policies, Threat Insights), Monitor (Observability), Plan (Product Planning), AI-powered (Duo Chat, Duo Workflow, AI Framework, AI model validation, Custom models)
Pini Wietchner	Senior Product Manager, AI-powered:AI Framework
Suzanne Selhorn	Staff Technical Writer, AI-powered (Duo Chat, AI Framework, AI model validation, Custom models)

Team members who are supporting this team on a short-term basis are located here where “Work priorities” include “AI Framework.”

☕ Team Responsibilities

Team responsibilities include

• Facilitating the integration of AI capabilities throughout GitLab by leveraging the AI Abstraction Layer and AI Gateway.
• Guaranteeing the presence of comprehensive global observability, monitoring, documentation, and enhancements in latency.
• Providing essential support for our AI chat system framework.
• Incorporating support for new AI providers when required.
• Assisting with the production support and ensuring the readiness of the AI Gateway.
• LLM inference support, including prompt enginering, response evaluation, fine tuning, evaluation, and more.

☎️ How to reach us

Depending on the context here are the most appropriate ways to reach out to the IDE Group:

• Slack Channel: #g_ai_framework
• Slack Groups: @ai-framework (entire team) and @ai-framework-engs (just engineers)

📦 Team Processes

📆 Regular team meetings

❗️Important: For every meeting, the AI Framework team’s meeting document should be used, and filled with the meeting notes, as well as references to any other sync meeting agendas/notes/recordings which have recently occurred. This will make it easier for people to find any meeting notes.

Team Meetings

1. Weekly Refinement Assignment Meeting
   • When: Every Monday, alternating between 14:00 AM GMT+1 and 19:00 PM GMT+1
   • What: This meeting replaces the previous Work Assignment Meeting and focuses on refining issues. The Engineering Manager and Product Manager ensure all issues are properly refined.

Shared calendars

• AI Framework PTO (Calendar ID: c_eca9440729dba2cbd88b3117fa70839836fb5811cb072132b94c52f912a31bf5@group.calendar.google.com)
• AI-Powered Stage Calendar (Calendar ID: c_n5pdr2i2i5bjhs8aopahcjtn84@group.calendar.google.com)

AI Framework team members should sync your PTO events with AI Framework PTO calendar.

🖖 Weekly EM Updates

Each week the team EM provides a Weekly Status update issue which aims to capture the most important items for the team to be aware of. These can be found here.

📚 AI Framework Board Outline

Our workflow process involves a weekly board walk where we review issues with the Deliverable label. Here’s how we organize our board:

How do people know what to work on?

An issue is ready to be taken by an assignee when it has all of the following:

• Deliverable label
• The current Milestone
• Either workflow::ready for development or workflow::refinement label

Board Lists

1. Open 📝: This list includes all identified issues. An engineering manager will be assigned if either the Milestone or the “workflow::ready for development” label is missing.
2. workflow::design ✏️: During this phase, issues undergo design refinement. After design considerations are integrated, the “ready for development” label is applied.
3. workflow::refinement 🧪: Issues in this stage are undergoing engineering refinement to ensure the proposed solution meets all requirements. Once refined, the “ready for development” label is applied.
4. workflow::ready for development 🎯: Issues that are prioritized and assigned to a specific milestone are moved to this list, and the “ready for development” label is applied.
5. workflow::in dev 👩‍💻: When a developer begins work on an issue, they should move it to this list and apply the “in dev” label.
6. workflow::in review 👀: After development is complete, the issue moves to this list, and the “in review” label is applied.
7. workflow::verification ✅: Following a successful code and UX review, the issue should be moved to this list and the “verification” label should be applied.
8. workflow::complete 🎉: Once the issue is verified and confirmed to be working properly, it should be moved to this list, the “complete” label should be applied, and the issue should be closed.

🔄 Processes

🗓️ Weekly Process

1. Open 🆕: (DRI: PM) Capture and list new issues as they arrive. This stage serves as the initial holding area for issues awaiting classification and prioritization.
2. Workflow Design ✍️: (DRI: Designer) Move issues into this lane for design refinement, focusing on user experience and interface design.
3. Workflow Refinement 🔧: (DRI: Engineering Lead) Conduct technical refinement to ensure feasibility and adherence to standards.
4. Workflow Ready for Development 🎯: (DRI: PM/EM) Transition fully refined and specced issues here, ready for developers to start coding.
5. Workflow in Development 💻: (DRI: Assigned Developer) Implement and build the specified features and functionalities.
6. Workflow in Review 🔍: (DRI: QA/Reviewer) Perform thorough reviews and testing, including code reviews and usability testing.

📝 Issue Guidelines

These guidelines apply to all issues we use for planning and scheduling work within our group. Our Engineers can define specific implementation issues when needed, but the overall goal for our issues are as follows:

• Provide a meaningful title that describes a deliverable result.
  • ✅ Add the new GitLab Duo chat package as a Vue2 extension
  • ✅ Chat: move away from using OpenAI embeddings
  • ❌ Make Chat better
• Provide a meaningful description that clearly explains the goal of the issue, and provide some technical details if necessary.
• Should there be critical implementation steps or other useful ways to create small tasks as part of the issue, please use a checklist as part of the issue descriptions.
• The issue should have the Deliverable label, milestone and workflow label assigned.
• Design and frontend engineering use one issue. The same issues moves from workflow::design to workflow::refinement to workflow::ready for development. This ensures that there is a single source of truth for customer-facing issues. If a design issue is too large to be implemented, it may be promoted to an epic.

It’s okay to create specific engineering-driven implementation issues for more complex features. These would be called Child Issues and they should always link back to their parent. If one issue would spawn many child issues, consider creating an Epic.

🏋 Weighting and Estimation Process

New OKR Implementation (3-Milestone Learning Period)

• Duration: 3 milestones
• Goal: Implement and refine a lightweight weighting process
• Ownership: Issue assignee is the weight owner
• Process:
  • Initial weight assignment when issue is created
  • Weight updates as understanding evolves
  • Process review after 3 milestones

Weight Guidelines

Issues are weighted using the Fibonacci sequence (0, 1, 2, 3, 5, 8, 13+):

• Weight 0: Smallest issues (typos, minor formatting, simple code changes without tests)
• Weight 1: Simple issues with minimal uncertainty (good for new contributors)
• Weight 2: Straightforward issues requiring multiple code/test updates
• Weight 3: Larger issues with some complexity but manageable scope
• Weight 5: Should typically be broken down; acceptable for large manual updates with low risk
• Weight 8/13+: Placeholder weights indicating need for breakdown; too large or uncertain for immediate implementation

Weight Update Process

1. Initial weight assigned upon issue creation
2. Assignee owns responsibility for weight accuracy
3. Weights can be adjusted as understanding of complexity evolves
4. Team reviews weight accuracy during milestone retrospectives

This new process will be evaluated after 3 milestones to assess effectiveness and make necessary adjustments.

📝 AI Feature Evaluations Guidelines - Evaluate like I am 5

See the Evaluate like I am 5 Project and read the docs here.

👏 Communication

The AI Framework Team communicates based on the following guidelines:

1. Always prefer async communication over sync meetings.
2. Don’t shy away from arranging a sync call when async is proving inefficient, however always record it to share with team members.
3. By default communicate in the open.
4. All work-related communication in Slack happens in the #g_ai_framework channel.

⏲ Time Off

Team members should add any planned time off in the “Workday” slack app, so that the Engineering Manager can use the proper number of days off during capacity planning.

🤙 Ad-hoc sync calls

We operate using async communication by default. There are times when a sync discussion can be beneficial and we encourage team members to schedule sync calls with the required team members as needed.

🔗 Other Useful Links

🌍 AIGW Region Deployments

• 🚀 Staging
• 🌐 Production

📝 Dashboards (internal only)

• Requests per provider
• Error budgets
• AI Gateway SLIs
• GCP quota limits
• LLM Sidekiq completions
• Duo Chat Question Categorization
• Security Issues
• Reliability Issues
• Sentry via CompletionWorker
• Sentry via Feature Category
• Monthly Retros
• Chat QA Evaluation
• Chat REST API Error Ratio

📹 GitLab Unfiltered Playlist

The AI Framework Group collates all video recordings related to the group and its team members in a playlist in the GitLab Unfiltered YouTube channel.

View page source - Edit this page - please contribute.