Code Suggestions Model Rollout Guide
This document serves as a guide for rolling out Code Suggestions models.
Create a rollout plan
Create an issue using the Rollout Plan Template. This must be done before or during the implementation of the new model. Specific rollout plans may need to be created for different rollout phases.
Rollout method
Rollout of a new model will be done through a feature flag.
You must create a feature flag rollout issue for the new feature flag.
Pre-rollout checklist
- Make sure that you have reviewed the considerations when adding a new model.
- Make sure that the model provider or host can handle the volume of requests on AI Gateway.
Rollout phases
Rollout of a new model is usually done in 3 phases:
- Rollout to Code Creation team members and other interested stakeholders.
- Rollout to all GitLab team members.
- Rollout to all users.
Roll out to Code Creation team members and other interested stakeholders
This is an optional but recommended step before rolling out to all GitLab team members. Code Creation team members and interested stakeholders will have more knowledge of the currently used models, and they can give valuable feedback around latency and correctness in comparison to current models.
Before rollout, make sure that an internal feedback issue has been created. Make sure to direct all feedback to the internal feedback issue.
Roll out to all GitLab team members
This is a required step before rolling out to all users. With dogfooding from GitLab team members, the larger volume of requests can help us determine if:
- We need to increase the request quota for the model.
- Additional post-processing is needed for the completions coming from the model.
- We need to reach out to the model providers for bug fixes and other considerations.
Before rollout, make sure that the internal rollout and internal feedback issues have been created. Make sure to direct all feedback to the internal feedback issue.
Roll out to all users
This step is done when the model is ready for widespread use. Rollout to all users can be done by percentage of actors (recommended), or you can roll out to all users at once.
Since this is considered a feature release, ensure proper announcement of the release through appropriate channels before launch.
d2462d25)
