Growth Fast Boot September 2019

Growth groups met in September 2019 for a fast boot to discuss, plan, and begin implementing a Growth strategy at GitLab

The Acquisition, Expansion, Conversion and Retention groups took part in a Fast Boot in September 2019. The planning issue contains the proposal for Fast Boot, and this page documents our discussions and outcomes.

Day 1

Link to Day 1 Agenda Document

Key Outcomes

How we operate as a team

Section Level

  • Monthly Product/Growth metrics review with E-Group (VP Product leads, with support from Growth Director)
  • Every 6 weeks Growth Group Conversation (PM + Eng Director lead)
  • Weekly engineering meeting
  • Weekly ideation & prioritization meeting
  • On Monday, check in on experiments to identify which can be concluded, and to collect data for review meeting on Tuesdays

Group Level

  • Groups will organize/meet as needed

How do we run Growth experiments?

  • Development board labels/process - continue on & follow handbook
  • A/B testing process, artifacts, expectations
  • Each experiment will set confidence level - 95%-99%
  • Test summary written by data analyst in the issue (add a “a/b test complete” label for searching later)

Expectations for working in another teams’ code base

  • Emphasize that the point is to gather data, not ship production code
  • Challenge: review process too slow. Reviewers vs Maintainers. Only a small number of maintainers.
  • Goal: one deliverable picked up per week

Feature Ownership

Fulfillment owns underlying billing, licensing, and transactional system, as well as current portal user experience.

  • Acquisition owns the New Signup ARR growth KPI + the new signup UX
  • Conversion owns the New Free to Paid Trial ARR growth KPI + the trial UX
  • Expansion owns the Current customer seat expansion + current customer upgrade ARR growth KPI + the upgrade flow & true-up flow
  • Retention owns the Gross Retention KPI + renewal flow

Growth PM’s will own issues for key flows; engineering leaders will help assign work

Long Term: the vision is for the fulfillment team to own the billing and licensing platform, with a set of API’s and SLA’s, so the growth teams can build and curate user experiences on top of them to drive great customer experiences (and resulting KPI’s).

Deep Dives

Teardown/journey map existing flow. Leverage demos, videos, screenshots. Ask “experts” to join remotely to demo and share their pain points/opportunities. Session to be led by the product manager.

Acquisition Overview

Discussion led by Jensen, PM for Acquisition, topics covered include:

  • KPIs and mission for the Acquisition team
  • Initial areas of focus
  • Teardown of the existing experience
  • Teardown of the competitors experience
  • Identifying opportunities for improvement

Video

Conversion Overview

Discussion led by Sam Awezec, PM for Conversion, topics covered include:

  • KPIs and mission for the conversion team
  • Initial areas of focus (Initial product activation, Ah-ha moments with paid features and/or limits, Trial experience and overall value)
  • Teardown of the existing experience
  • Identifying opportunities for improvement

Video

Expansion Overview

Discussion led by Tim Hey, PM for Expansion. Topics covered:

  • Mission
  • KPIs & Supporting Performance Indicators
  • Customer facing opportunities
    • User Orientation - Users don’t know where to start
    • Increase platform confidence and trust - I love my tools and am afraid to switch
  • Internal opportunities
    • Self-Managed usage for upsell process and user workflow
    • Self-Managed True-up process and user workflow

Video

Slides

Retention Overview

Discussion led by Mike Karampalas, PM for Retention. Topics covered:

  • KPIs for retention
  • Near-term and longer-term focus for the Retention team
  • Teardown of the renewal flow(s)
  • Opportunities for improvement

Video

Slides

Day 2

Link to Day 2 Agenda Document

Data Deep Dive

Discussion led by Eli Kastelein, Growth Data Analyst. Topics covered:

  • GitLab.com data vs Self-Managed data
  • Overview of data sources relevant to the growth team
  • Vision for how we can use data to serve the growth team
  • Event-based reporting versus database reporting
  • What’s currently blocking us from collecting the data we need to be successful

Video

Slides

State of Product Intelligence

During this time we walked through Product Intelligence as a team and reviewed where we were headed. The main item we covered was the Vision Page in the handbook and how we need to clearly communicate what product analytics is and where to find more information.

Topics covered (which will also be added to the handbook)

  • Vision
  • SMAU – what it is and how it’s managed
  • Dashboards – what they are, self-serve and how to request one
  • Data fields available for tracking and what to do if you need a new one
  • Tracking and instrumentation – how to’s for GitLab.com and Self-Managed
  • Technologies we use

MR under review

A/B Testing Requirements

Issue: Implement an A/B testing solution that can measure the impact of Growth experiments

Key outcome is to work with the Release team to improve our existing Feature Flags capability in GitLab in order to successfully A/B test Growth experiments.

Right now in order to run an experiment, we need to do additional work to ensure we can measure the results. This work will be bespoke for each experiment while we improve our capability.

We can investigate third party solutions if needed.

Days 3-5

At this time the individual Growth teams split into their groups to create and work on issues in their respective areas.

Fast Boot Outcomes

Implemented new team names, KPI’s and objectives

Streamlined team processes

Create and deliver issues

Acquisition

Expansion

Conversion

Retention