Runner Core Bug Triage Decision Matrix

The RICE-Adapted Priority Score

Formula

Priority Score = Reach + Customer Impact + Community Signal + Workaround Factor

Range: 0-10 points

Scoring Components

1. Reach (0-3 points) - Executor Impact

Based on Strategic investment:

2. Customer Impact (0-3 points)

3. Community Signal (0-2 points)

4. Workaround Availability (0-2 points)

Decision Actions by Score

We can adjust this to be more aggressive after a first round of triage and trying it out with bugs open in the current milestone.

Scenario

Bug Wranglers may close bugs when a combination of these conditions are met:

• Bug affects docker+machine or other deprecated feature
• Low Impact - Priority Score <= 4
  • No customer escalation
  • Bug age > 1 Year (~60% of our current backlog are bugs over a year old)
• Behaviour is documented
• Cannot reproduce after 2 attempts
• Reporter pinged for more info and no response within 7 days

Immediate Closure Conditions

Applying to Current Backlog

Here are examples based on our current backlog:

Likely Schedule

• Error with recursive submodule checkout (fatal: transport ‘file’ not allowed) - (24 upvotes, Tier 1 executor)
• GitLab Runner build failures for Docker deployments (Docker 29) - (19 upvotes, Tier 1 executor)
• Jobs fail despite successful task completion when pods are evicted (since Runner 17.9) - (18 upvotes, Tier 1 executor)

Close

• Docker-machine Preparation Failed
  • I closed this and there’s been minimal pushback (1 comment) despite it having 22 upvotes and 8 disappointed emojis
• Gitlab Runner with docker+machine keeps spawning ec2 instances but is unable to use them
  • I closed and there’s been no pushback despite it having 23 upvotes
• Fails to stop spot instance
  • I closed this and there’s been no pushback despite it having 10 upvotes
• UTF-8 and GBK encoding bug in gitlab-runner + virtualbox + pwsh
  • Closed: Reported in Runner v15.1.0 with no evidence of preproduction in a recent version
• 403 forbidden during helper image pull
  • Close! First reported on v15.4.0 with no significant traction since then.

Sample Copies for Closing

• General low priority/score

Thank you for reporting this issue and for your patience.

After evaluating this bug against our backlog, we've determined it falls below our current prioritization threshold. This means the impact is limited relative to other issues in our backlog, a workaround exists or low related reports. We're also constantly working on broad architecture improvements to Runners which could make this issue obsolete in the future.

This is not a judgment on the validity of your report. We simply have finite capacity and must focus on issues with the broadest impact.

If circumstances change (wider impact or condition changes making this more urgent), we're more than happy to revisit.

/status "Won't do"

• Can’t reproduce

Thank you for reporting this issue.

We attempted to reproduce this on Runner {{current_version}} but were unable to confirm the issue. This may indicate the bug was resolved in a subsequent release, is environment-specific or additional context is required to reproduce. See [maintained versions](https://docs.gitlab.com/policy/maintenance/#maintained-versions).

If you're still experiencing this issue, please let us know by confirming the issue occurs in our current Runner version with additional information to help us reproduce this.

We're happy to revisit once we can reliably reproduce the behavior.

/status "Won't do"

• Deprecated docker+machine

Thank you for reporting this issue.

This bug affects the `docker+machine` executor, which was deprecated in GitLab 17.5 and is scheduled for removal.

We are no longer investing engineering effort in fixes for deprecated executors. We recognize this may be frustrating if you're still relying on this functionality. Please see our [documentation](https://docs.gitlab.com/runner/executors/) for other executor options that may work for you.

We appreciate your understanding as we focus resources on supported executors.

/status "Won't do"

Research

This draws from industry best practices like:

• GitLab’s Severity/Priority Matrix:
  • https://handbook.gitlab.com/handbook/product-development/how-we-work/issue-triage/#outdated-issues
  • “We simply can’t satisfy everyone. We need to balance pleasing users as much as possible with keeping the project maintainable” - https://handbook.gitlab.com/handbook/product-development/how-we-work/issue-triage/#lean-toward-closing
  • “After 14 days, if no response has been made by anyone on the issue, the issue should be closed” - https://handbook.gitlab.com/handbook/product-development/how-we-work/issue-triage/#outdated-issues
  • “Bug fix backports are maintained for the current (first) version…” - https://docs.gitlab.com/policy/maintenance/#maintained-versions
• RICE Scoring Model (Intercom): Reach, Impact, Confidence, Effort
  • https://www.intercom.com/blog/rice-simple-prioritization-for-product-managers/
• Kubernetes Triage Lifecycle:
  • https://www.kubernetes.dev/docs/guide/issue-triage/#bugs
  • Example: https://github.com/kubernetes/test-infra/issues/25967#issuecomment-1285025238
• Chromium Won’t Fix Categories:
  • https://www.chromium.org/chromium-os/developer-library/reference/bugs/life-of-a-bug/#no-action-issues

View page source - Edit this page - please contribute.