Incident Review Bar Raiser
This page covers the Bar Raiser role in detail: what the role is, how reviews are assigned, and how to get involved. For the incident review process itself, including the other roles and the completion criteria, see the Incident Review page.
What a Bar Raiser is
The Bar Raiser panel reviews all incident reviews. A review must have Bar Raiser approval before it can be closed.
The Bar Raiser upholds high standards in our incident reviews in the same way maintainers uphold standards in our code. A high level overview of the role is in the Incident Review Bar Raiser section.
What a Bar Raiser is not
- Not the author. The Bar Raiser does not write the review, fill in the timeline, or create the corrective actions.
- Not the schedule owner. Missing the review SLO is the DRI’s responsibility, not the Bar Raiser’s.
- Not an approver of record for the fix. Approval means the review meets our standard for understanding and corrective actions, not that the Bar Raiser has verified the code or configuration changes themselves.
- Not a gate on blame. Reviews are blameless. A Bar Raiser who sees blame in a review should ask for it to be reframed.
Process
Finding a Bar Raiser
Bar Raisers are assigned on a rotation using the Bar Raiser on-call schedule, which has a Primary and a Secondary rotation.
To find the Bar Raiser for a review:
- Visit the Bar Raiser on-call schedule and request review from the Primary.
- If the Primary is out of office, use the Secondary.
- If both are out of office, ping
@gitlab-com/gl-infra/incident-review-bar-raisersin #incident-review-bar-raisers to request help.
The rotation exists so that the work is shared evenly and so that a DRI never has to guess who to ask. It is not an on-call obligation in the way an incident rotation is: reviews are asynchronous, so being Primary does not require immediate response, and it should not affect vacation time or other commitments. When you are away, the Secondary and the wider panel cover for you.
Requesting a Bar Raiser review
The DRI should seek a review from the Bar Raiser as soon as the incident review is initially drafted so the BR can provide feedback quickly. The DRI requests review by ‘at’-mentioning the Bar Raiser on the incident review issue.
Giving a bar raiser review
Bar Raisers give feedback in comments on the incident review issue so that the discussion stays with the review. When the review meets our standard, the Bar Raiser records their approval on the issue, which unblocks the DRI to close it.
The Bar Raiser shall:
-
Strive to have a first response within 12h of a mention/ask for review - including a response to ask other Bar Raiser’s for help.
-
Read the review with fresh eyes and ask the questions that have not been asked yet, or re-ask questions that haven’t been answered deeply enough
-
Probe whether the contributing causes are genuinely understood, not just described
-
Challenge whether the corrective actions address the real problem, and whether they are sized and prioritized appropriately
-
Push back when a review is incomplete, superficial, or leaves obvious threads unpulled
-
Approve the review when it meets our standard, unblocking the DRI to close it
The Guidance for conducting a review section lists the questions a Bar Raiser is likely to ask, including how corrective actions should match the type of root cause. Those questions are the shared standard for both roles: the DRI should be considering them while writing, and the Bar Raiser should be checking them while reviewing.
How to become a Bar Raiser
Being a bar raiser is open to anyone with interest in building a culture of operational excellence and reliability for our customers. If you want to help raise the quality of our incident reviews:
- Join #incident-review-bar-raisers in Slack.
- Ask an existing Bar Raiser in that channel to add you as an Owner of the incident-review-bar-raisers group.
- When you are ready to start taking on reviews, ask Steve Abrams to add you to the Primary and Secondary rotations in the Bar Raiser on-call schedule.
A good way to build context before joining is to read recent incident reviews, follow the discussion in #incident-review-bar-raisers, and attend or watch recordings of the Weekly Incident Review Meeting, where recent high severity incidents are discussed while they are fresh and reviews may still be in progress.
78fd06b1)
