GitLab Incident Communications Plan

Escalations, Processes, and How to Manage Incidents

If you need review from the corporate communications team on a potential incident, gather information in an issue or link us to the Slack discussion by message the team in the #corpcomms Slack channel and we’ll work with the Security Communication Manager on Call to determine next steps

Incident Communication Plan

We use our incident plan and communication escalation process enables the entire organization to understand actions/inactions to take when an incident occurs. Consider it like “tiers”, where when a specific tier is communicated to the team members, all appropriate departments know what actions to take/not take.

What is an Incident?

A contentious issue that undermines a critical attribute of our company’s brand and challenges its reputation. And/or an incident which impacts, or has the potential to impact the safety and well-being of our team-members.

Defining the scope/severity of a potential incident

When determining the need for communication escalation, it is important to first determine what the immediate issue is and determine what caused said issue. After collecting this information, review Tier 1-4 to determine the severity and subsequent action plan

Tier 4 - Event

A Tier 4 event is something we already have a process for like a security incident or a customer support request via Zendesk. In the case of a Tier 4, please follow the established company process.

Tier 3 - Rapid Response

A Tier 3 is a rapid response needed to address industry changes, competitive moves, news about the company, and other opportunities to provide comment/opinion or thought leadership. In the case of a Tier 3, please follow the Rapid Response directive.

Tier 2 - Incident (process below)

A Tier 2 incident is a current issue, gray area communication or planned sensitive company communication that could lead to loss of customers, team members or market trust in the company. An incident has short term impact, but is high risk. Examples include; hiring policy, implementing unwelcome product changes, executive changes, being disingenuous to company values. If it meets the criteria for an incident, please initiate the below process.

Tier 1 - Escalated Incident

A Tier 1 escalated incident is a significant reputational or financial risk to the company with long-term impacts. Industry examples include class-action lawsuits, highly public litigation (IP infringement or sexual harassment lawsuits), board removal of executives, etc. Please follow the Escalated Incident response team and process.

Tier 2 Process

If a Tier 2 communication escalation arises, please use Slack or a text message to alert:

  1. The Head of Corporate Communications and Internal Communications
  2. If the former is unavailable, the Head of Corporate Marketing and CMO

The head of corporate communications will propose a recommendation on how to best proceed with an external company message and recruit the resources to accomplish the plan (this becomes the #1 priority for each resource, unless physically impossible or their manager provides a replacement). A template for a Tier 2 communication escalation action plan can be found here.

The head of corporate communications will assess the request within 1 HOUR (6amPST-6pmPST). Urgency will be assessed and will be determined to be 3 HOURS (ASAP) or 24 HOURS turnaround, and the communication escalation action plan will be scoped accordingly - with a bias towards releasing an MVC as soon as possible, and iterating as more news becomes available.

Core Communication Escalation Response Team Roles and Responsibilities

  • Corporate Communications phone tree - This team will coordinate external communications efforts according to the communication escalation action plan and liaise across the extended GitLab teams to ensure all parties are activated, updated and aligned. This team is also responsible for social media.
  • Internal Communications - This team will coordinate internal communications efforts according to the action plan.
  • Developer Relations phone tree - This team is responsible for community response situations, monitoring, and enforcing the GitLab Code of Conduct on the relevant GitLab.com discussions.
  • CLO - The Chief Legal Officer will provide legal input on the action plan, as well as all written external and internal communications.
  • CMO - DRI on final external message. Any disagreements on urgency or action will be escalated immediately to the CMO for a final decision.

Extended team roles, responsibilities and points of contact

Social Media

Monitor Across Channels with Social Listening (All Tiers)

While monitoring takes place on a regular (if not several times a day) basis across social channels from both the Developer Relations team and the social team, it’s particularly important to monitor channels on a more frequent basis during this time. There is a need to have many instigators (that may be different per channel) and it’s critical to try and understand this information in order to best evaluate the actions to be taken for communication escalation.

Social Listening could take place in the existing GitLab brand health monitor inside of Sprout Social. Because listening is always on (and Sprout backfills data, it is not an immediate need to set up a new topic.

What a tier 2 incident looks like on organic brand social channels

Tier 2 incidents could either be addressed publicly without further interruption or be cause for a temporary pause on outbound organic social media from our brand channels. In either scenario, inbound responses (replying to folks directly rather than a new message from GitLab) should be included as part of the response. Generally, Developer Relations and support teams use canned messages for product service interruptions to communicate with customers and the community, especially if the incident is product or service-related.

When the incident is not caused by our product and is not service-related, the need to coordinate with support teams may not exist. However, working with the Developer Relations team helps to address core and niche social media channels.

To learn more about pausing social media / going dark during an incident, review the social media handbook.

Community Response

The Community Operations Manager will work through existing frameworks for tiers 3 and 4. For a tier 2 incident, the director of Developer Relations and the community operations manager will be looped into the existing e-group led response.

For tier 2 incidents, the community team follows their responsible portions of the comms-reactive-tier-2-incident-checklist template.

Communicating Internally with Key Stakeholders

Coordinating communication between people working on the communication escalation action plan.

Create Coordination Slack Channels
  • A communication escalation coordination channel with all DRIs and stakeholders. The traffic here will be low to medium volume and focused on coordination, mitigation and strategy. Invite one person designated as the representative from the core team listed above, as well as, a representative of each specific area of business that is central to the incident. RADCIE will inform our value of Collaboration.
  • A communication escalation discussion channel, open to anyone at GitLab. Everyone at GitLab can provide their input, and some discussions or topics will be escalated to the incident coordination channel. The traffic here will be high volume and focused on enabling GitLab team members visibility on particular topics or developments related to the incident.

All Slack channels related to the incident should contain a link to the issue and a link to the communication escalation action plan in their channel description

Create a New Issue For Each Incident

Open a new issue in the corporate marketing project using the comms-reactive-tier-2-incident-checklist template. Be sure to confirm that the issue is marked as confidential, as it pertains to work prior to releasing communications. Use the issue as our project center, assigning to-do’s and accomplishing tasks. Updates including links to external messages from the community should also be added as a comment in the issue, we should look to reduce the volume and locations of chatter on the topic into a singular location, since we will always need to corral several slack threads to the issue.

Our values guide our actions: our sense of urgency under Results and our value of not waiting under Iteration and RADCIE will under our value of Collaboration.

Internal Response

Where possible when dealing with an incident, we want team members to hear about it internally first. Depending on the incident though this may not be possible. However as soon as communication has happened externally we should aim to respond quickly internally - even if it’s just to acknowledge we are aware/and are preparing a statement/ or looking into the details of the issue and will update team members as soon as we have more information.

Internal Communications Channel

  • #Company-FYI
  • #diversity_inclusion_and_belonging
  • #external-comms

Communicating Externally

Centering Communications on GitLab Values + Research - In “Incorporating Social Media in Risk Communication,” it’s recommended that we handle communication escalation through:

A number of these recommendations directly or indirectly align with GitLab’s Values.

Additionally, we need to be sure we:

Meet Stakeholders Where They Are

Communications intended to alleviate the incident should be delivered to owned and earned channels alike.

Only addressing an incident in a blog post, and telling the community and the media to click a link may work, but that demands moving into our arena first. GitLab should be comfortable with hybrid communications - e.g. updates via blog, but turning the email into a threaded Tweet where the first tweet also includes a link to the blog. Other places to consider posting or adding a link include: GitLab forum, footer of Support and Sales emails, and Help Center docs.

In order to have maximum reach, we’ll need to consider alleviating the incident for the community at every possible step in their journey with GitLab communication (some folks would click the blog link right away, others would follow an issue comment - in both cases we captured a higher reach because we gave both options).

Make messages distinguishable from one another, depending on the audience. Consistency does not need to be homogenous.

The specific makeup of this mix will change depending on the incident.

Don’t Over Communicate

It’s important to use listening (either a tool or manual research) to understand where the incident is emerging. More often than not, most incidents will originate on 1 to 3 channels (HackerNews, Twitter, Reddit, GitLab issue, news article, etc.), and it is entirely possible to manage the incident by communicating on these same channels only.

If an incident is not properly analyzed, GitLab runs the risk of over communicating and inviting the issue to be more severe.

Decide Which Communication Will be the Last to Go Out

At some point on managing an incident, planned external communication is to be thought of as our “last response” to our stakeholders and the greater GitLab community. This is identified when the company feels that we’ve addressed everything we can in the communication, we’ve taken ownership of any mistakes made, and we’ve committed to our stance.

Once we’ve identified this last communication, it is best to stop addressing the issue in the public square. No more broadcasts to stakeholders across any other channels. Always direct those with follow up questions to our last communication. In most “small” to “medium” incidents, the last communication may be our first communication.

After the Incident

Determine that the incident is actually over

We’ll know when the heat starts to die down naturally as our inboxes and notifications across channels begins to return to normal levels. However, we shouldn’t go by instinct only. In most cases, we should use a listening tool or review @mentions and non @mentions across channels to quantify the incident being over, e.g. determine that the volume of messages are decreasing and/or sentiment* returning to what we determine is a normal level.

*depending on the tool, sentiment analysis can be an inaccurate view of actual sentiment. In some cases, manual tweaking may be necessary to accurately capture.

Create a retrospective

Once we determine that the incident is over, move to reporting mode. Narrate the story of Why the incident happened, where and when it took place, who was affected by the incident initially and are there any long-term effects, Who was talking about the incident externally, what the team did, and How the incident was resolved and How we can prevent this from happening in the future.

It’s important to include data analysis and “the story” of stakeholders, community members, and GitLab team members. Neither the numbers nor “the story” express the entire picture. Use screenshots of positive and negative feedback from the community.

Optional: Deliver the retrospective issue to the entire team after feedback and updates are made, and invite all GitLab team members to a post mortem AMA to discuss more about the incident and invite an open Q&A. Always be sure to close this meeting with a link to the final talking points in the event GitLab team members need to answer this incident in their own conversations.

After retrospective is shared, archive designated incident Slack channels.

(Optional) Public Retrospective- In the case of a severe product- or service-related incident, publishing a summary of the communication escalation process should be considered. A summary of the incident in the days or weeks following could include scope, impact, timeline, root cause, resolution, and planned improvements.

Stick with the Final Talking Points

As discussed when deciding which communication will be your last, it’s best to finalize your communications and end new talking points. We’re likely to still be asked about the incident even after it’s over.

It is important to stick to the final talking points and direct everyone to our last communication. If this is a blog, a tweet thread, or a moment in a YouTube video, we should be ready to direct those that ask.

Last modified November 4, 2024: Fix broken links (2eb0e162)