Test Platform in Distribution group

Overview

The goal of this page is to document existing Quality Engineering activities in Distribution group.

Dashboards

• QE Distribution dashboard - Dashboard to track Quality Engineering work items
• Distribution Issues - Dashboard to visualize metrics important for Bug Prioritization
• Bug Prioritization metrics - Bugs metrics required for Bug Prioritization (ensure to filter by Distribution group)

Quality work

Quality work is being tracked in epic#9057. The epic lists large initiatives that need to be worked on to better support quality in Distribution group.

GitLab QA

GitLab QA is being used in several Distribution projects to validate that GitLab works as expected.

Project	Tests type	Schedule
GitLab Omnibus	Full	QA mirror pipeline is triggered
GitLab Charts	Sanity	Run automatically in merge requests and scheduled against default branch
GitLab Charts	Full	Triggered manually in merge requests
GitLab Operator	Smoke	Run automatically in merge requests
GitLab Operator	Full	Manually triggered
Reference Architecture Tester	Full	Manually triggered and FIPS QA Nightly

Check Running GitLab QA for information on how to run GitLab QA locally for development.

Investigate QA failures

1. Search for the failure in open Pipeline Triage issue or search for the spec name in the main GitLab project
   • If Allure report is available: Click on report link -> Product defects -> Select failed spec -> click Failure issues. Demo
   • Some specs might have multiple QA failure issues with different stack trace. In such case, compare failed stack trace from the job with the ones listed in the issues.
2. If an issue with the same error is not found
   • Continue to debug the QA failure following the guide
   • Reach out to the Test Platform sub-department - on-call DRI or Distribution SET

Bug Prioritization

The Distribution team works together on Bug Prioritization and aims to close at least 6 bugs per milestone based on the team’s current availability. The number of bugs per milestone is revisited in the following issue#1100.

Process:

• Team creates a new Planning issue
• SET creates a new issue using Bug Prioritization template
• SET reviews open bugs using Distribution Issues
  • Add Severity labels to bugs that are missing a severity label
  • Review open bugs following Prioritization Guidelines
• SET to propose in team planning issue 6 bugs to be considered in milestone
• At the end of the quarter:
  • SET reviews Distribution Issues metrics for open bugs
  • SET shares analysis with the Distribution team
  • The Distribution discusses if the process should be adjusted

Quad Planning

Quality team reviews open issues for quad planning following Quad Planning process.