Operate Workflow

Overview of how work is performed by Operate team.

Overview

The Operate team is responsible for installation, upgrades, scaling, migration, and configuration across various deployment methods. Our work often involves:

  1. Responding to both internal and external customer requests
  2. Supporting multiple deployment methods and platforms simultaneously
  3. Managing technically complex integration work with varied timelines
  4. Balancing planned feature development with maintenance work

Following Kanban approach provides these benefits:

  • Improved flow efficiency - Work items progress continuously rather than being tied to milestone boundaries
  • Greater adaptability - Easier to respond to urgent deployment issues or customer requests without disrupting planned work
  • Better visualization of workload - Clearer view of in-progress work across different deployment platforms
  • More predictable delivery - Work completion is based on team capacity rather than milestone deadlines

Kanban Implementation

We also use the GitLab product development flow and labels in principle. However, we usually skip below phases due to the nature of our work:

Workflow Diagram

flowchart LR
    classDef backlog fill:#f9f9f9,stroke:#999
    classDef refinement fill:#fce8d4,stroke:#e67e22
    classDef ready fill:#d4f6c8,stroke:#2ecc71
    classDef development fill:#d4e8f6,stroke:#3498db
    classDef readyreview fill:#b3d1f0,stroke:#2980b9
    classDef review fill:#e0d4f6,stroke:#9b59b6
    classDef done fill:#e8e8e8,stroke:#666
    classDef blocked fill:#ff9999,stroke:#e74c3c,stroke-width:2px,stroke-dasharray: 5 5

    Backlog([backlog]):::backlog --> Refinement([workflow::refinement]):::refinement
    Refinement --> Ready([workflow::ready for development]):::ready
    Ready --> Development([workflow::in dev]):::development
    Development --> ReadyReview([workflow::ready for review]):::readyreview
    ReadyReview --> Review([workflow::in review]):::review
    Review --> Done([workflow::complete]):::done

    Blocked([workflow::blocked]):::blocked
    Refinement -.-> Blocked
    Ready -.-> Blocked
    Development -.-> Blocked
    ReadyReview -.-> Blocked
    Review -.-> Blocked
    Blocked -.-> Refinement
    Blocked -.-> Ready
    Blocked -.-> Development
    Blocked -.-> ReadyReview
    Blocked -.-> Review

    LookLeft["Look Left<br>(Daily)"] -.-> Refinement
    LookRight["Look Right<br>(Daily)"] -.-> ReadyReview
    LookRight -.-> Review
    WeeklySync["Weekly Sync<br>(Weekly)"] -.-> Backlog
    WeeklySync -.-> Ready
    WeeklySync -.-> Development
    BlockedReview["Blocked Issues Review<br>(Daily Priority)"] -.-> Blocked

    subgraph WIP Limits
        Refinement -.- RL["2 per engineer"]
        Ready -.- RDL["1.5× team size"]
        Development -.- DL["2 per engineer"]
        ReadyReview -.- RRL["2× team size"]
        Review -.- RVL["1.5× team size"]
    end

    subgraph Ceremonies
        BlockedReview

        LookLeft
        LookRight
        WeeklySync
    end

Kanban Board Structure

The Operate Kanban board would use the following columns, aligned with GitLab’s product development workflow:

Column Description WIP Limit
backlog Prioritized but not ready for development No limit
workflow::refinement Issues being refined, broken down 2 per engineer
workflow::ready for development Fully specified, ready to be worked on 1.5× team size
workflow::in dev Currently being implemented 2 per engineer
workflow::ready for review Implementation complete, waiting for reviewer to pick up 2× team size
workflow::in review Under active code review 1.5× team size
workflow::complete Completed work No limit
workflow::blocked Work that cannot proceed due to some reason Tracked but no limit

Key Ceremonies

Daily Activities

  • Look Left & Right Before Dev - Help move one item forward from refinement or review (ready for review/in review) before starting new development
  • Async Stand-up - Brief asynchronous check-in focused on flow and impediments

Weekly Meetings

  1. Team Sync (Weekly)

    • Review Kanban board progress with EM and PM
    • Discuss impediments and prioritization
    • Ensure alignment with product goals

  2. Team Demo (Weekly)

    • Sharing with Build group
    • Regular demonstrations of completed work
    • Sharing knowledge about new features or improvements
    • Gathering feedback from team members and stakeholders
    • Increasing visibility of the team’s accomplishments

  3. Maintainer Discussion (Weekly)

    • Sharing with Build group
    • Regular review of merge requests and code quality
    • Discussion of technical decisions and architecture
    • Coordination between maintainers on critical components
    • Ensuring consistency in code standards and practices

  4. Engineers Huddle (On demand)

    • Development-focused discussion
    • Implementation strategies and technical decisions

Monthly Activities

On the last week’s weekly sync meeting every month.

  • Flow Retrospective - Review flow metrics and process improvements
  • Release Planning - Align with next milestone planning using flow-based priorities

Quarterly Activities

  • Roadmap planning - Participate in quarterly roadmap planning to align flow with strategic objectives
  • Retrospective - Reflect on the past quarter’s achievement, identify areas for enhancement, and create action plans for the future

Priority definition

Operate team uses infrastructure-wide priority labels to determine the position in the Kanban columns.

Priority Label Position in Kanban Action Required
1 ~priority::1 Top of active columns Immediate attention required
2 ~priority::2 High in active columns Address after Priority 1 items
3 ~priority::3 Middle of workflow Address after higher priorities
4 ~priority::4 Bottom of workflow Address when resources available

  • “Active columns” refer to the workflow stages where work is currently being processed. These include:

    • Ready for Development: Issues prioritized and ready to be worked on
    • In dev: Issues currently being worked on by team members
    • Ready for review: Work completed but waiting to be reviewed
    • In review: Work completed but going through review and approval

  • “Middle/Bottom of workflow” refers to issues with ~priority::3 and ~priority::4 that are positioned in the central part of our board. These items:

    • Have moderate urgency
    • Are positioned below Priority 1 and 2 items in active columns
    • Are typically scheduled after more urgent work
    • May be in any workflow stage (refinement, development, review)
    • Represent important work that should be completed, but can wait until higher priorities are addressed

  • The position in Kanban board needs to be managed and maintained manually due to the issue boards’ limitation.

Work Prioritization Guidance

  1. Highest Priority First: Always address higher priority items before any lower priority ones
  2. Milestone/Deadline-Driven Work: Issues with fixed deadlines or milestone take precedence over non-deadline work of the same priority
  3. Dependency Resolution: Address blocking issues before dependent issues regardless of initial priority
  4. WIP Limits: Maintain work-in-progress limits to ensure focus on completing high-priority items

Issue Weight Definition

Weight Extra investigation Surprises Collaboration Description Examples
1: Trivial not expected not expected not required Cannot benefit from further breakdown
2: Small possible possible possible Simple tasks with clear requirements
3: Medium likely likely likely More complex tasks requiring coordination
5: Large highly likely highly likely highly likely Consider breaking down if technically feasible
8+ guaranteed guaranteed guaranteed Too large - must be broken down into multiple issues and grouped into an Epic
  1. Notes on weights:
    • Weights are contextual and can be affected by domain knowledge, experience levels, and time at GitLab
    • Weights are not set in stone and can be adjusted by the author/assignee if an issue requires more effort than initially estimated
    • For Weight 5 issues, team members are encouraged to discuss whether breaking it down would be beneficial
    • Weight 8+ issues must be broken down and should not be marked as ready for development
  2. WIP Limits Implementation
    • Set column WIP limits based on item count initially, review once there is more data

Flow Metrics

Use these flow-based metrics instead of milestone-based completion metrics, we will establish the baseline metrics and aim for steady improvement by FY27Q1.

graph LR
    classDef metric fill:#d4e8f6,stroke:#3498db

    subgraph Kanban Metrics
        CT[Cycle Time]:::metric
        LT[Lead Time]:::metric
        FE[Flow Efficiency]:::metric
        TP[Throughput]:::metric
        WIP[WIP Ratio]:::metric
        WIA[Work Item Age]:::metric
    end

    subgraph Timeline
        B[Backlog] --> R[Refinement] --> RD[Ready] --> D[Development] --> RV[Review] --> C[Complete]
    end

    CT -.-> D
    CT -.-> C
    LT -.-> B
    LT -.-> C
    FE -.-> Timeline
    TP -.-> C
    WIP -.-> D
    WIP -.-> RV
    WIA -.-> Timeline

1. Cycle Time

Definition: The total elapsed time from when work begins on an issue until it is delivered to production.

Components:

  • Code Time: Time spent coding the solution
  • Review Time: Time spent in code review (MR open → MR merged)

Target: Our target cycle time varies by work complexity (To be updated by FY27Q1):

  • Trivial changes (Weight 1): < x
  • Small changes (Weight 2): < x
  • Medium changes (Weight 3): < x
  • Large changes (Weight 5): < x

Measurement: (To be updated by FY27Q1)

2. Lead Time

Definition: The total elapsed time from when an issue is created until it is delivered to production.

Components:

  • Planning Time: Time from issue creation to development start
  • Cycle Time: (as defined above)

Target: Our target lead times are (To be updated by FY27Q1):

  • Priority 1 issues: < x
  • Priority 2 issues: < x
  • Priority 3 issues: < x
  • Priority 4 issues: < x

Measurement: (To be updated by FY27Q1)

3. WIP Ratio

Definition: The ratio of work items in progress compared to team capacity. Calculated as the number of active work items divided by the number of team members.

Target :

  • Optimal WIP ratio: 1-3
  • Warning threshold: > 3.0
  • Critical threshold: > 4.0

Measurement: Count of issues in the ~“workflow::in dev”, and ~“workflow::in review” stages divided by the number of active team members.

4. Throughput

Definition: The number of work items (issues) completed per time period.

Target: Teams should establish their baseline throughput and aim for steady improvement.

Measurement: Count of issues moved to the “closed” state per week.

5. Work Item Age

Definition: The age of currently open work items.

Target:

  • No Priority 1 issues older than x days
  • No Priority 2 issues older than x days
  • Average work item age decreasing over time

Measurement: Current date minus issue creation date for all open issues.

6. Flow Efficiency

Definition: The percentage of time that work items are actively being worked on versus waiting.

Target: > x% flow efficiency (industry standard is often 15-20%)

Measurement: Active work time (time spent in (~“workflow::refinement” + ~“workflow::in dev” + ~“workflow::in review”)) divided by total lead time.

Required labels

Besides the GitLab product development flow labels, there are a number of additional required labels applied at any time to Epics, issues and merge requests (items):

  • group::Operate - Items specific to, or authored by us.

There are also a number of additional required labels under certain scenarios:

  • spike - Issues which primarily involve research to understand options and the breakdown of future deliverables. Spikes are often the first issue in a new Epic where the output defines additional issues and order of serial/parallel work.

In addition to the labels outlined above, see also workflow labels used during merge request review and labels used during issue triage.

Milestone Integration

Maintain alignment with GitLab milestone practices through:

  • Tag any strict timeline issue with appropriate milestone during creation and keep it till completion, e.g. breaking changes
  • Keep the milestone as Next 1-3 releases for all other issues when actively working on them
  • Keep all other issues with milestone Backlog
  • Tag all completed work with the milestone for recording
  • Maintain all standard required labels

Blocked Issues Management Process

Blocked Issue Tracking

  • ~“Workflow::blocked” Lane - Issues that cannot progress due to dependencies, waiting for input, or other impediments are moved to a dedicated “workflow::blocked” lane
  • Blocker Documentation - Each blocked issue must include a comment describing:
    • What is blocking the issue
    • Who/what team is responsible for unblocking
    • Any actions taken to resolve the blockage
    • Expected timeframe for resolution (if known)

Blocked Issue Review (Daily Priority)

  1. Daily Blocked Issues Triage
    • At the start of each day, the team reviews all blocked issues
    • Blocked issues are sorted by priority and duration of blockage
  2. Escalation Path
    • Issues blocked for less than 2 days: Issue assignee actively follows up
    • Issues blocked for 2-5 days: Team sync/async discussion and EM involvement
    • Issues blocked for 5+ days: Escalation to appropriate managers/departments
  3. Resolution Actions
    • Issue assignee is responsible for following up on each blocked issue
    • Document all actions taken to resolve the blockage
    • Consider workarounds or scope adjustments if blockage cannot be resolved quickly
  4. Metrics and Reporting
    • Track percentage of issues blocked vs. flowing
    • Report on average time to resolve blockages
    • Identify patterns in blockers to address systemic issues
  5. WIP Management for Blocked Issues
    • Blocked issues still count against workflow::in dev WIP limits
    • Team may temporarily adjust capacity allocation when multiple issues are blocked
    • Consider swarming on blocked issues to remove impediments

Unblocking Workflow

flowchart TB
    classDef normal fill:#d4e8f6,stroke:#3498db
    classDef action fill:#d4f6c8,stroke:#2ecc71
    classDef blocked fill:#ffd4d4,stroke:#e74c3c

    IB[Issue Becomes Blocked]:::blocked --> |Move to| BL[workflow::blocked Lane]:::blocked
    BL --> |Document| BC[Blocker Cause]:::action
    BL --> |Set| ET[Expected Timeframe]:::action

    BL --> |Daily Review| DR[Daily Blocked Issue Review]:::normal
    DR --> |Can Proceed?| D{Decision}

    D -->|Yes| UI[Unblock Issue]:::action
    D -->|No| AE{Age Evaluation}

    AE -->|< 2 days| FU[Team Member Follows Up]:::action
    AE -->|2-5 days| TE[Team Discussion & EM Involvement]:::action
    AE -->|> 5 days| ES[Escalate to Senior Management]:::action

    FU --> DR
    TE --> DR
    ES --> DR

    UI --> |Move back to original column| WF[Workflow Continues]:::normal
    UI --> |Document resolution| DR[Document Resolution]:::action
Last modified July 29, 2025: [Docs] Rename self managed team to operate team (35575083)
View page source - Edit this page - please contribute. Creative Commons License