How we measure team productivity

The productivity metrics some DevOps engineering teams track, their targets, and how they are calculated.

Summary

Plan:Planning Views, Plan:Portfolio Planning and Create:Source Code Hardening and Modernization track four metrics on the team MR throughput dashboard. Every number is a team aggregate, per engineer on a 28-day moving window and adjusted for time off. Per-person data is never shown.

Metric What it counts Target
MR rate Merged MRs authored by the team, per engineer per 28 days 15 to 30
Review balance Reviews the team gave to others minus reviews it received from others Above 0
Time to merge Median days from marked ready to merged, for authored and for reviewed MRs 0.5 to 1.5 days
Open MRs Open MRs per engineer at the end of the day At most 2

Metrics

MR rate

Merged MRs authored by team members in tracked projects, per engineer per 28 days, adjusted for time off.

Target: 15 to 30. The target is a band, not a line. Slightly below 20 is not underperforming, and reaching 20 is not the place to stop.

Review balance

Reviews team members gave on other people’s MRs, minus reviews other people gave on the team’s MRs, per trailing 28 days. Reviews within the team cancel out. Bot reviewers such as GitLab Duo Code Review are not counted.

Target: above 0. Every MR needs reviewers, so a team should review at least as much for others as others review for it. Community and bot-authored MRs have no team behind them, so the balance has to be positive, not zero.

Time to merge

Median days from the MR being marked ready for review to it being merged. Time in draft does not count. The median is used because a few long-lived MRs dominate the mean.

Two series are tracked:

  • Authored: MRs authored by team members. How fast the team gets its own work merged.
  • Reviewed: MRs team members reviewed. How quickly the team acts as a reviewer and helps other people’s work get merged.

Target: 0.5 to 1.5 days for both. Time to merge moves together with MR rate, and it is the less contentious of the two: nobody argues against having their work merged faster.

Open MRs

Open MRs at the end of each day, the subset marked ready for review, and the median age of open MRs.

Target: at most 2 per engineer.

How it is calculated

  • Moving window. Each day shows the trailing 28 days, so the effect of a change is visible within days rather than at the end of the next calendar month. The dashboard also offers 7 and 84 day windows; targets scale with the window.
  • Adjusted for time off. MR rate and review counts are divided by the share of working days the team was available, using absences recorded in Workday, including public holidays. Time to merge and open MRs are not adjusted.
  • Team membership. Each person’s current team in Workday is applied to the whole history, because Workday records team moves weeks to months after they happen.
  • Tracked projects. MRs are counted in the projects that make up the product. The list is TRACKED_PROJECTS in the dashboard source.

Exact definitions, the tracked-project list and all settings are in the dashboard source and its README, and in the “How this is measured” section at the bottom of the dashboard.

Dashboard

The team MR throughput dashboard runs on Streamlit in Snowflake and requires Snowflake access. Any team in the Engineering directory can be selected and compared side by side.