Test Platform in Distribution group

Overview

The goal of this page is to document existing Quality Engineering activities in Distribution group.

Dashboards

• QE Distribution dashboard - Dashboard to track Quality Engineering work items
• Distribution Issues - Dashboard to visualize metrics important for Bug Prioritization
• Bug Prioritization metrics - Bugs metrics required for Bug Prioritization (ensure to filter by Distribution group)

Quality work

Quality work is being tracked in epic#9057. The epic lists large initiatives that need to be worked on to better support quality in Distribution group.

GitLab QA

GitLab QA is being used in several Distribution projects to validate that GitLab works as expected.

Project	Tests type	Schedule
GitLab Omnibus	Full	QA mirror pipeline is triggered
GitLab Charts	Sanity	Run automatically in merge requests and scheduled against default branch
GitLab Charts	Full	Triggered manually in merge requests
GitLab Operator	Smoke	Run automatically in merge requests
GitLab Operator	Full	Manually triggered
Reference Architecture Tester	Full	Manually triggered and FIPS QA Nightly

Check Running GitLab QA for information on how to run GitLab QA locally for development.

Investigate QA failures

1. Search for the failure in open Pipeline Triage issue or search for the spec name in the main GitLab project
   • If Allure report is available: Click on report link -> Product defects -> Select failed spec -> click Failure issues. Demo
   • Some specs might have multiple QA failure issues with different stack trace. In such case, compare failed stack trace from the job with the ones listed in the issues.
2. If an issue with the same error is not found
   • The Distribution team owns triaging and debugging its QA failures; continue to debug the failure following the guide
   • As a last resort for severe issues, reach out to Developer Experience in #s_developer_experience

Bug Prioritization

The Distribution team works together on Bug Prioritization and aims to close at least 6 bugs per milestone based on the team’s current availability. The number of bugs per milestone is revisited in the following issue#1100.

Process:

• Team creates a new Planning issue
• A Distribution team member creates a new issue using Bug Prioritization template
• A Distribution team member reviews open bugs using Distribution Issues
  • Add Severity labels to bugs that are missing a severity label
  • Review open bugs following Prioritization Guidelines
• The Distribution team proposes in its team planning issue 6 bugs to be considered in the milestone
• At the end of the quarter:
  • A Distribution team member reviews Distribution Issues metrics for open bugs
  • The findings are shared with the Distribution team
  • The Distribution discusses if the process should be adjusted

Quad Planning

Quality team reviews open issues for quad planning following Quad Planning process.