Import Group

The Import Group facilitates migrations.

About

The Import group is a part of the Create Stage. The group supports the product by migrating between GitLab instances and from other providers.

This page covers processes and information specific to the Import group. See also the group direction page and the features we support per category.

How to reach us

To get in touch with the Import group, it’s best to create an issue in the GitLab project and add the ~"group::import" label, along with any other appropriate labels. Then, feel free to ping the relevant Product Manager and/or Engineering Manager.

For more urgent items, feel free to use the Slack Channel (internal): #g_import.

Team Members

The following people are permanent members of the group:

Name Role
Carla DragoCarla Drago Senior Backend Engineer, Create:Import
George KoltsovGeorge Koltsov Senior Backend Engineer, Create:Import
James NuttJames Nutt Senior Backend Engineer, Create:Import
Oiza BaiyeOiza Baiye Frontend Engineer, Create:Import
Reza MarandiReza Marandi Backend Engineer, Create:Import
Rodrigo TomonariRodrigo Tomonari Staff Backend Engineer, Create:Import
Sam WordSam Word Senior Backend Engineer, Create:Import
Thiago FigueiróThiago Figueiró Manager, Software Engineering, Create:Import

Work

A week before each milestone, the Engineering Manager creates a planning issue with the priorities for the team. The team members are notified, and proceed to update the issue with their commitments for the milestone.

Issue Development Workflow

We use the standard GitLab engineering workflow.

Issue Boards

The work for the Import group can be tracked in the following locations:

Issue Labels

To increase discoverability, apply the correct labels to issues.

All issues should have:

  • ~"group::import" (the bot will apply stage and section labels accordingly)
  • One or more of the category labels:
    • ~"Category:Importers" (FIXME: at the moment the bot forces all issues to have this category)
    • ~"Category:Webhooks"
  • A type label
  • A workflow label
  • ~"backend" or ~"frontend", if appropriate

For issues related to importers, also apply an Importer: label. For example: ~"Importer:GitHub" or ~"Importer:Direct Transfer".

Backlog Refinement

Engineers typically refine the issues they plan to work on. However, anyone may refine an issue to make it ready for development, especially if a community contributor might pick it up.

When an issue is part of an Epic, the DRI may refine it or delegate refinement to an engineer assigned to that Epic. Regardless of who refines the issue, aim to complete refinement before the milestone begins.

At the latest, refinement should start when the Engineering Manager shares the planning issue for the next milestone, which happens a week before the end of the current milestone.

Identifying Issues for Refinement

The Engineering Manager schedules issues, which are then included in the milestone planning issue.

Based on each engineer’s allocation for the milestone, identify issues that are not in Ready for Development status. These typically have Refinement or Planning breakdown status, but may have any Product Development Flow status.

Issues that are Ready for Development but were refined many months ago should be refined again with a focus on changes in the codebase, product or architecture direction.

Refining Issues

Follow the Create stage Cross-Team Planning and Refinement guidelines, specifically the Refinement and Implementation Plan sections.

For the Import group, an issue is considered refined when it meets those guidelines plus the following Import-specific requirements:

  • Weight (optional for type::bugs)
  • Peer review by another engineer (optional for weight 1 issues); to show it’s been reviewed, the reviewer can leave a comment or move the issue to Ready for development
  • Ready for development status

Bug readiness

Bugs do not need to be perfectly understood before we work on them, and hence don’t need a weight.

The effort to fully understand a bug is often most of the effort of fixing it. Consequently, proposed solutions in bug issues may be considered suggestions made on an imperfect understanding of the defect.

Try to include at least:

  • Steps to reproduce
  • Current behavior
  • Expected correct behavior

The Bug template includes these fields.

Requesting help

If you are not part of the Support organization, we recommend reaching out to them first, as they have greater availability and can assist with most common issues. There’s a dedicated Slack channel #spt_pod_import you can join, follow and ask questions in. However, there are times when in-depth technical knowledge is needed to resolve a customer issue, requiring the involvement of an engineer from the team.

Before requesting help from the Engineering team, please first review the GitLab documentation for the topic of your interest and the additional resources listed below:

If you cannot find the answer to your question in the resources listed above, please open a Request for Help (RFH) issue and use the SupportRequestTemplate-Import template. Please ensure that you provide all the required information before reaching out to the team; otherwise, we will be unable to proceed with your request. New issues will be prioritized according to our internal triage process. Please note that we can only support requests for issues affecting the current and the two most recent minor GitLab versions (N-2). We cannot offer a fix for older versions. This is aligned with our maintenance policy for backports.

Milestone Doctors

Each milestone, two backend engineers in the team are assigned the role of Milestone Doctor, one as Primary and the other as Secondary. The assignments are in the Milestone Doctor rotation schedule. Engineers can freely trade upcoming shifts and update the schedule.

The Secondary doctor steps in when the Primary is OOO or over capacity, assuming the same responsibilities. Otherwise, they work on tasks planned for the milestone. The Primary doctor should let the Secondary know when they’re OOO or facing capacity issues.

The milestone capacity for the primary Doctor is 100% assigned to the role responsibilities. If there are none left, the engineer should work on ~type::maintenance or ~type::bug issues, and consider increasing their code review capacity.

The current doctors can be tagged with @gitlab-com/create-team/import/reaction-rotation.

Responsibilities

  • Engage with Support and PS on new RFH issues
  • Follow-up on long-lasting open issues
  • Assist the Support team on customer calls
  • Maintain team runbook documentation on how Milestone Doctors have successfully diagnosed problems
  • Respond to questions in our team Slack channel #g_import
  • Update our FAQ with any learnings from the shift
  • Review importer dependencies, and create issues for required changes due to 3rd-party API changes.
  • Update membership of @gitlab-com/create-team/import/reaction-rotation at the end of the milestone.
Importer dependencies

Once per milestone, review each importer’s dependency changelogs for upcoming breaking changes and API deprecations. Use GitLab Duo Chat to assess impact, and create an issue with the relevant ~"Importer:" label for any change that requires an update.

After the review, leave a comment in the planning issue with a summary and links to any issues created; tag the EM for attention.

Suggested prompt

Replace [START_DATE] with the last date when the dependencies where checked.

You are helping an engineer on the GitLab Import team perform a periodic review of third-party API dependencies. Our importers integrate with external services and we need to check their changelogs for any breaking changes, deprecations, or required migrations announced since [START_DATE].

Review the following changelog pages and identify any changes that could break or require updates to our importers:

**GitHub importer**
- Source: https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/github_import
- Changelogs: https://docs.github.com/en/rest/about-the-rest-api/breaking-changes, https://github.blog/changelog/, https://github.com/octokit/octokit.rb/releases

**Bitbucket Cloud importer**
- Source: https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/bitbucket, https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/bitbucket_import
- Changelogs: https://developer.atlassian.com/cloud/bitbucket/changelog/

**Bitbucket Server importer**
- Source: https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/bitbucket_server, https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/bitbucket_server_import
- Changelogs: https://developer.atlassian.com/server/bitbucket/changelog/

**Gitea importer**
- Source: https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/import/gitea_controller.rb (uses GitHub import client)
- Changelogs: https://github.com/go-gitea/gitea/blob/main/CHANGELOG.md, https://blog.gitea.com

**Jira importer**
- Source: https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/jira_import
- Changelogs: https://developer.atlassian.com/cloud/jira/platform/changelog/, https://github.com/sumoheavy/jira-ruby/releases

**FogBugz importer**
- Source: https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/fogbugz_import
- Changelogs: https://support.fogbugz.com/section/3113-articles

For each change you find, check the corresponding GitLab implementation to verify whether the change affects us.

Then, for each importer, list:
1. The name of the importer
2. Whether changes are required
3. A summary of the required change (for changes that do not affect us, simply provide a link to
   the announcement)
4. The due date or enforcement date

If no actionable changes are found, confirm that and note the date range you checked.

Working with Security

The group has an existing threat model to assist in identifying issues that may have security implications, but there are other considerations.

An Application Security Review should be requested when the issue or MR might have security implications. These include, but aren’t limited to, issues or MRs which:

  • falls under the threat model
  • handles binary files (downloading, decompressing, extracting, moving, deleting)
  • modifies or uses file manipulation services
  • uses methods from Import/Export CommandLineUtil

Longer lived feature flags

This is a supplement to GitLab’s common development guidance for use of feature flags. It applies to all flag types besides the ops type.

Changes to Import features often happen in high-traffic code paths and have led to outages on GitLab.com in the past. Outages are often to do with resource contention that can be difficult to see ahead of time in code review or in QA testing.

  • Large imports can trigger thousands of workers.
  • Integrations and webhooks are executed millions of times a day.
  • Contention problems sometimes do not surface immediately, and only when large customers trigger the new code path.

For this reason we should prefer to keep feature flags in the codebase for a longer period of time than normal. During this time the flag is enabled by default but can still be disabled quickly in the event of an incident.

In the past, we were able to quickly mitigate several incidents by disabling the feature:

For changes within importers, integrations or webhooks we should prefer to:

  1. Roll out the flag with /chatops as normal.
  2. QA the changes using large data to proactively flush out any problems at scale. For importers, see our runbook for tips.
  3. When you come to release the feature, change the feature flag to be default_enabled: true rather than to remove it. This is the optional release the feature with the flag step on the flag rollout issue.
  4. At this point the feature is considered released within the milestone, and can be announced in the release post as it will ship to self-managed customers.
  5. Wait between 1-3 weeks where the flag exists in the codebase but remains enabled by default. Use the longer period for changes in areas of more contention, or if you feel it may take longer to detect problems for any reason.
  6. After that period, remove the feature flag to complete the flag rollout process.

During a release

  • When an issue is introduced into a release after Kickoff, an equal amount of weight must be removed to account for the unplanned work.
  • Development should not begin on an issue before it’s been estimated and given a weight.
  • By the 15th, engineering merge requests should be merged. In other words, we assume code merged after the 15th will not be in the release. That allows time for the release to be finalized, and any associated release posts to be merged by the 17th. (This is an experiment starting with 13.11.)

Release posts

For issues which need to be announced in more detail, a release post can be automatically created using the issue. When working on an issue, either in planning, or during design and development, you can use the release post item generator to have the release post created and notify all the relevant people.

If you do not want an issue to have a release post, make sure that the issue does not have a release notes section or do not use a release post item:: label.

Proof-of-concept MRs

We strongly believe in Iteration and delivering value in small increments. Iteration can be hard, especially when you lack product context or are working on a particularly risky/complex part of the codebase. If you are struggling to estimate an issue or determine whether it is feasible, it may be appropriate to first create a proof-of-concept MR. The goal of a proof-of-concept MR is to remove any major assumptions during planning and provide early feedback, therefore reducing risk from any future implementation.

  • Create an MR, prefixed with PoC:.
  • Explain what problem the PoC MR is trying to solve for in the MR description.
  • Timebox it. Can you determine feasibility or a plan in less than 2-3 days?
  • Identify a reviewer to provide feedback at the end of this period.
  • Close the MR. Provide a summary in the original issue on what you learned from the PoC, including product and performance implications.
    • State whether you are able to move forwards with implementation or not.
    • Please do not close the issue.

The need for a proof-of-concept MR may signal that parts of our codebase or product have become overly complex. It’s always worth discussing the MR as part of the retrospective so we can discuss how to avoid this step in the future.

Retrospectives

We have 1 regularly scheduled “Per Milestone” retrospective, and can have ad-hoc “Per Project” retrospectives.

Per Milestone

The Import group conducts milestone retrospectives in GitLab issues. These include the engineers, UX, PM, and all stable counterparts who have worked with that team during the milestone.

Participation by our team members is highly encouraged for every milestone.

These are confidential during the initial discussion, then made public in time for each month’s GitLab retrospective. For more information, see group retrospectives.

Per Project

If a particular issue, feature, or other sort of project turns into a particularly useful learning experience, we may hold a synchronous or asynchronous retrospective to learn from it. If you feel like something you’re working on deserves a retrospective:

  1. Create an issue explaining why you want to have a retrospective and indicate whether this should be synchronous or asynchronous.
  2. Include your EM and anyone else who should be involved (PM, counterparts, etc).
  3. Coordinate a synchronous meeting if applicable.

All feedback from the retrospective should ultimately end up in the issue for reference purposes.

Tech Leads

Our group works with tech leads to help organize work on different topics and identify DRIs for them.

Characteristics of a Tech Lead

A tech lead is:

  • an individual contributor with additional responsibilities. Every engineer regardless of their seniority is qualified to be a tech lead.
  • a temporary role that is tied to a specific topic/project. We allow the team to have multiple tech leads at the same time for different topics/projects.
  • not a manager.
  • not an additional seniority level.

The Tech Lead role provides growth opportunity for engineers who are interested in adopting leadership skills.

Responsibilities of a Tech Lead

Tech leads wear many hats. Their responsibilities may differ from project to project but may include:

  • Technical Vision and Architecture - Defining and evolving the overall technical architecture for a given project
  • Technical Guidance - Providing technical guidance and mentoring to other developers on the team
  • Planning and Prioritizing Work - Organizing the work by breaking down bigger tasks into smaller actionable items
  • Tracking Progress - Tracking progress on commitments and reporting status updates
  • Risk Management - Identifying, assessing and managing technical risks that may impact deliverables
  • Coordination - Overseeing the work of others and helping remove blockers
  • Technical documentation - Maintaining documentation of the technical architecture and code structure for other developers

Current Tech Leads

Below is an overview of topics that are overseen by a tech lead:

Topic Tech Lead Topic Link Notes
Direct Transfer - User contribution mapping Rodrigo Tomonari Epic -
Improve the efficiency of developer contributions to the importers James Nutt OKR -
Congregate tbd https://gitlab.com/gitlab-org/gitlab/-/issues/428657
GitHub Actions tbd https://gitlab.com/gitlab-org/manage/general-discussion/-/issues/17652

Merge request roulette reviews

When areas of the Import codebase are changed, the reviewer roulette will recommend that the merge request is reviewed by an Import team member. This will only happen when the merge request is authored by people outside of the Import team. See this example of how the review recommendation looks.

The reasoning behind these special recommendations is that other groups have some ownership of certain integrations or webhooks. Reviewing changes made by non-team members allows us to act as owners of foundational code and maintain a better quality of the Import codebase.

How roulette matches work

File paths of changes in a merge request are matched against a list of regular expressions. The roulette uses these hash values to recommend reviewer groups. For example, :import_integrate_be and :import_and_integrate_fe will recommend Import backend and frontend reviews respectively. As the regex matches are first match wins and not cumulative, any other relevant reviewer groups like :backend or :frontend must also be included in each hash value.

The regex list should be updated to match integrations or webhooks code whenever needed. The list matches our commonly namespaced files, so new code in existing namespaces will always match.

To see which files in the GitLab repository produce a match, paste the following in a Rails console:

require Rails.root.join('tooling/danger/project_helper.rb')

ALL_FILES = Dir.glob('**/*');

def category_regexs(category)
  matching_categories = Tooling::Danger::ProjectHelper::CATEGORIES.select do |regexs, categories|
    next if regexs.is_a?(Array)

    Array.wrap(categories).include?(category)
  end

  regexes = matching_categories.map(&:first)
  Regexp.union(*regexes)
end

def print_files(category)
  regex = category_regexs(category)

  puts ALL_FILES.grep(regex).reject { |path| File.directory?(path) }.sort
end

puts "Backend:\n"
print_files(:import_integrate_be)

puts "Frontend:\n"
print_files(:import_integrate_fe)

Monitoring

This is a collection of links for monitoring our features.

Grafana dashboards

Sentry errors

Kibana dashboards

See a list of all Import Kibana dashboards.

Importer dashboards:

API/Webhooks dashboards:

Kibana logs

GitLab for Jira Cloud app workers:

Error budgets

GitLab uses error budgets to measure the availability and performance of our features. Each engineering group has its own budget spend. The current 28-day spend for the Import team shows in this Grafana dashboard.

Error budget spend happens when either of the following exceeds a certain threshold:

  • Error rate of an endpoint or worker
  • Apdex (latency) of an endpoint

Determine the highest-impact fixes

To determine the highest-priority problems in our Grafana dashboard:

  1. Go to the Error budget panel.
  2. Expand Budget spend attribution. The Budget failures panel is ordered by top failures.
  3. In Failure log links, click the corresponding links.

Fixing the top offenders will have the biggest impact on the budget spend.

Further resources

Learn more about error budgets with these resources:

Usage data dashboards

You can view data for feature usage in Tableau.

Last modified May 7, 2026: Update Import work section (daf7a289)
View page source - Edit this page - please contribute. Creative Commons License