Product usage playbooks with usage metrics
Using metrics to trigger product usage playbooks and leading discovery with data
GitLab collects product usage metrics via service ping from self-managed customers. For each of the GitLab use-cases, we have at least a single usage metric reported in the service ping. These metrics are used to trigger Product Usage playbooks in Gainsight to prompt CSMs to reach out to customers in a consultative context to help the customer adopt GitLab features as use-case solutions and maximize their return on investment with GitLab. Below is a list of the metrics used to trigger each of the current product usage playbooks. Furthermore, there is information defining the triggers for each playbook and discovery questions that CSMs can use to guide the consultation process with the customer.
CI/CD Use Case
| Metric | Triggers | Definition | Discovery |
|---|---|---|---|
ci_runners |
Less than one pipeline, one month after onboarding | Total configured runners on GitLab instance | This number staying at zero implies that the customer does not have any GitLab runners configured hence not using GitLab CI. It is important to understand the customer’s reasoning behind this; do they not use CI/CD in their workflow all together or does GitLab not meet their needs? |
ci_internal_pipelines |
Drops by at least 20% month on month or remains below 50% of projects on GitLab instance three months after onboarding |
Total pipelines in GitLab repositories | A drop in this number shows a lessening use of CI pipelines. That in itself might not be necessarily a bad thing as this could also happen as a result of projects consolidating to run fewer efficient pipelines instead. Understanding the reason behind the change can help guide the CSM’s next steps. |
ci_external_pipelines |
Higher than ci_internal_pipelines month on month for three months |
Total pipelines in repositories hosted outside of GitLab | Growth in this metric is suggestive of GitLab CI being used for an increasing number of projects outside of GitLab. This is likely during a migration between different SCM tools. To maximize the customers return on investment in GitLab CSM’s can use these numbers to understand how a migration is progressing and well as ensuring that GitLab’s source code management is meeting the customer’s needs. |
projects_jenkins_active |
Higher than 20% of projects on GitLab instance after onboarding or growth month on month | Count of projects with active integrations for Jenkins | Ideally this metric should trend down as CSMs help onboard customers onto GitLab’s single DevSecOps platform. If the metric is trending up instead, then it is important to understand the customer’s motivation and liaise with product to ensure that the customer can benefit from GitLab CI/CD. |
projects_teamcity_active |
Higher than 20% of projects on GitLab instance after onboarding or growth month on month | Count of projects with active integrations for Teamcity CI | Ideally this metric should trend down as CSMs help onboard customers onto GitLab’s single DevSecOps platform. If the metric is trending up instead, then it is important to understand the customer’s motivation and liaise with product to ensure that the customer can benefit from GitLab CI/CD. |
projects_bamboo_active |
Higher than 20% of projects on GitLab instance after onboarding or growth month on month | Count of projects with active integrations for Bamboo CI | Ideally this metric should trend down as CSMs help onboard customers onto GitLab’s single DevSecOps platform. If the metric is trending up instead, then it is important to understand the customer’s motivation and liaise with product to ensure that the customer can benefit from GitLab CI/CD. |
User Engagement Use Case
| Metric | Triggers | Definition | Discovery |
|---|---|---|---|
billable_user_count/license_user_count |
Drops by at least 10% month on month or remains below 50% a month after onboarding | Ratio of active users versus licensed users | A ratio less than one implies that the customer is not fully utilizing their license which is usually the case during onboarding or a consolidation of different GitLab instances into one. If this number drops month on month it is important to understand the reasons behind the drop from the customer and ensure that GitLab is meeting their needs. |
Secure Use Case
| Metric | Triggers | Definition | Discovery |
|---|---|---|---|
container_scanning_job |
Less than 50 jobs, one month after onboarding | Count of Container Scanning jobs run | This number staying at zero implies that the customer(Ultimate tier) does not have any containers which need to be scanned. It is important to understand the customer’s reasoning behind this; Are building and deploying containers part of their workflow? Were errors found when including the container scanning template into their CI file? Conduct additional research to understand what is the cause of low adoption. |
dast_jobs |
Less than 50 jobs, three months after onboarding | Count of DAST jobs run | This number remaining below 5 implies that the customer(Ultimate tier) does not have many web applications which need to be scanned. Confirm if the customer understands how the DAST scanning capability works. Does the customer provision Review App environments? How does the customer currently uncover vulnerabilities that occur in their app environments? Conduct research to understand what is the cause of low adoption. |
dependency_scanning_jobs |
Less than one job, one month after onboarding | Count of Dependency Scanning jobs run | This number remaining below 5 implies that the customer(Ultimate tier) does not have many projects where dependency scanning is necessary. It is important to understand the customer’s reasoning behind this; Do key projects rely on dependencies? Were errors found when including the dependency scanning template into their CI file? Conduct research to understand what is the cause of low adoption. |
secret_detection_scans |
Less than 50 jobs, one month after onboarding | Number of Secret Detection security scans run | This number remaining below 50 implies that the customer(Ultimate tier) does not have many projects which need to be scanned for leaked secrets. It is important to understand the customer’s reasoning behind this; Has there been any attempt to enable secret detection? Are there any applications that require sensitive credentials in order to function? Conduct research to understand what is the cause of low adoption. |
Last modified September 19, 2024: Fix broken links (
38406a39)
