Monitoring

How do we monitor how we are doing?

The Create Stage uses logs as a type of asynchronous retrospective. When there is a metric that we are focusing on improving, we will take the additional step to begin logging behaviors related to that metric. We look at the metric, and ask ourselves, “What is the story behind this number?”

Stories lead to trends

Looking at metrics across a team, across the stage or across an organization begins to tell a story. Each metric is a breadcrumb. As you piece the story together, if you are lucky, you will recognize trends. Once you identify trends, you can begin making changes and measuring the impact of those changes. Those changes in turn may or may not change metrics or trends. It is a big experiment and an opportunity to learn and improve!

Throughput Log

The entire Dev Sub Department makes a monthly entry into a Throughput Log. Each Engineering Manager in Dev is responsible for updating this document (internal).

The Throughput Log includes the following information:

• Team Name
• MR Rate from two months ago
• MR Rate from last month
• A percentage that reflects how much the MR rate either increased or decreased comparing the two months
• Bulleted list of explanations to describe events / activities / processes that led to the change in the MR Rate

This log gives the Dev Director visibility across the entire sub department. This log gives the Director the transparency to enable them to act if necessary in support of all of the Individual Contributors that report up through the Director.

Days to Merge Log

The Create Stage Engineering Managers makes a monthly entry into their own confidential Days to Merge Log. Each Engineering Manager is responsible for analyzing all merge request that took eight days or longer to merge and analyze these merge request to determine if there are any lessons to be learned from them.

The Days to Merge Log includes the following information:

• Team Member
• Number of Days to Merge
• Merge Request (linked title)
• Is this related to the team processes? If yes, please explain.
• Is this related to the individual team? If yes, please explain.
• Is this an exceptional MR that could not have been broken down?

This log gives Engineering Managers more transparency into how their team is working. If there are common themes month after month, these themes can serve as a guide for areas for the Engineering Manager to focus on.

The Senior Engineering Manager gains visibility across the entire stage from this log. There might be opportunities for the Senior Engineering Manager to support the Engineering Managers in their efforts.