At GitLab Quality is everyone’s responsibility. The Test Platform sub-department’s mission is to be a world class team that enables successful development and deployment of high quality GitLab software applications with kaizen workflow efficiency, reliability, productivity.
The Test Platform sub-department does this by focusing on:
The Test Platform sub-department vision is to focus on customer satisfaction and enable GitLab to deliver faster and efficiently by supporting GitLab’s principle of Quality is everyone’s responsibility.
Integral parts of this vision:
GitLab has a Three-Year Strategy. Our Yearlies connect our 3 year strategy to our shorter-term quarterly Objectives and Key Results (OKRs). The sub-department direction is accomplished through these objectives (OKRs).
Our focus is to support our FY25 Yearlies. They can be found in the internal handbook.
Objectives and Key Results (OKRs) help align our sub-department towards what really matters. These happen quarterly and are based on company OKRs. We follow the OKR process defined here.
Here is an overview of our current Test Platform OKR.
Test Platform owns several tools which form a 3-prong trident for Self-Managed Excellence: the GitLab Environment Toolkit (GET), the GitLab Performance Tool (GPT), and the Reference Architectures (RA). Together, these tools support our broader strategy of cementing customer confidence and contributing to their ongoing success by ensuring their instances are built to a rigorously tested standard that performs smoothly at scale.
For more information, please visit our Self-Managed Excellence page.
Test infrastructure provides stability, dependability, and testing continuity for better planning and implementation. It gives the foundation for engineers to write their tests and an execution platform to execute them. By standardizing and streamlining software development, deployment, and maintenance processes, Test Platform can enable engineers to deliver and improve applications more efficiently, while reducing errors, improving consistency, and increasing speed.
Given rapidly evolving technologies and our drive to provide a world class experience for GitLab users, Test Platform sub-department strives to meet the increasing demands of efficient, intelligent test coverage and confidence at scale. We aim to test the right things at the right time. We focus on exploring several new testing types and visibility improvements to increase the actionability, speed, and sophistication of our various test suites.
Test Platform has been key to supporting prospect POVs and providing prompt, knowledgeable troubleshooting for external customers, while continuing to have a deep commitment to supporting our internal customers as well. We support our internal and external customers by:
To order to ensure platform is reliable, scalable, and secure, Test Platform sub-department can help with setting up dashboards for capturing test covearge, performance issues, and conducting capacity planning to ensure that the platform can handle the expected load.
Find relevant dashboards here.
As AI has evolved to be a foundational and transformational technology that can provide compelling and helpful benefits through its capacity to assist, complement, empower, and inspire people in almost every field of human endeavor, the Test Platform sub-department is looking into ways to boost efficiency and reduce cycle times in every phase of the software development lifecycle.
Test Platform Engineers are always available to provide technical support and expertise to development teams to solve complex technical challenges and ensure that applications are built to industry standards. This includes but not limited to:
Infrastructure Department structure is documented here.
graph TD
A[Test Platform sub-department]
A --> B(Self-Managed Platform team)
A --> C(Test Engineering team)
A --> D(Test and Tools Infrastructure team)
click A "/handbook/engineering/infrastructure/test-platform"
click B "/handbook/engineering/infrastructure/test-platform/self-managed-platform-team"
click C "/handbook/engineering/infrastructure/test-platform/test-engineering-team"
click D "/handbook/engineering/infrastructure/test-platform/test-and-tools-infrastructure-team"
Feel free to reach out to us by opening an issue on the Quality Team Tasks project or contacting us in one of the Slack channels listed below.
| Team | GitLab.com handle | Slack channel | Slack handle |
|---|---|---|---|
| Test Platform | @gl-quality/tp-sub-dept |
#test-platform | None |
| Self-Managed Platform team | @gl-quality/tp-self-managed-platform |
#self-managed-platform-team | @self-managed-platform |
| Test Engineering team | @gl-quality/tp-test-engineering |
#test-engineering-team | @test-engineering-team |
| Test and Tools Infrastructure team | @gl-quality/tp-test-tools-infrastructure |
#test-tools-infrastructure-team |
@test-tools-infrastructure |
| Name | Role |
|---|---|
 Vincy Wilson
|
Director, Test Platform |
 Abhinaba Ghosh
|
Engineering Manager, Test Platform, Test and Tools Infrastructure |
 Kev Kloss
|
Frontend Engineer |
 Ksenia Kolpakova
|
Engineering Manager, Test Platform, Test Engineering |
 Kassandra Svoboda
|
Manager, Quality Engineering, Core Platform & SaaS Platform |
 Nao Hashizume
|
Backend Engineer, Engineering Productivity |
 Peter Leitzen
|
Staff Backend Engineer, Engineering Productivity |
The following people are members of the Self-Managed Platform team:
The following people are members of the Test Engineering team:
| Name | Role |
|---|---|
|
Ksenia Kolpakova
|
Engineering Manager, Test Platform, Test Engineering |
 Désirée Chevalier
|
Senior Software Engineer in Test, Dev:Plan |
 Jay McCure
|
Senior Software Engineer in Test, Dev:Create |
 Joy Roodnick
|
Software Engineer in Test, Test Engineering, Verify:Runner group, Fulfillment section |
 Tiffany Rea
|
Senior Software Engineer in Test, CI:Verify |
 Valerie Burton
|
Senior Software Engineer in Test, Test Engineering, Fulfillment section |
 Will Meek
|
Senior Software Engineer in Test, Secure |
The following people are members of the Test and Tools Infrastructure team:
| Name | Role |
|---|---|
|
Abhinaba Ghosh
|
Engineering Manager, Test Platform, Test and Tools Infrastructure |
 Andrejs Cunskis
|
Senior Software Engineer in Test, Test and Tools Infrastructure |
 Chloe Liu
|
Staff Software Engineer in Test, Test and Tools Infrastructure |
 Dan Davison
|
Staff Software Engineer in Test, Test and Tools Infrastructure |
 David Dieulivol
|
Senior Backend Engineer, Engineering Productivity |
 Ievgen Chernikov
|
Senior Software Engineer in Test, Test and Tools Infrastructure, Analytics section |
 Jennifer Li
|
Senior Backend Engineer, Engineering Productivity |
 Mark Lapierre
|
Senior Software Engineer in Test, Test and Tools Infrastructure |
In addition to GitLab’s communication guidelines and engineering communication, we communicate and collaborate actively across GitLab in the following venues:
GitLab is an all-remote, timezone distributed company as such we optimize for asynchronous communication. While some topics benefit from a real-time discussion, we should always evaluate meetings to ensure they are valuable. We follow the guidance for all-remote meetings, including starting and ending on time - or earlier.
Group Conversations take the information from the Key Review (plus any additional topics) and shared with all of GitLab. All Team Members are invited to participate in Group Conversations by adding questions and comments in the Group Conversation Agenda.
Coordination of Infrastructure Group Conversation materials and facilitation of the discussion is a rotating role among the managers within the department.
Group Conversation DRI Schedule.
By the end of the week, we populate the Engineering Week-in-Review document with relevant updates from our department. The agenda is internal only, please search in Google Drive for ‘Engineering Week-in-Review’. Every Monday a reminder is sent to all of engineering in the #eng-week-in-review slack channel to read summarize updates in the google doc.
The Test Platform sub-department holds an asynchronous retrospective for each release. The process is automated and notes are captured here (GITLAB ONLY).
While this team operates as a several teams, we emphasize on ensuring the prioritization and needs of Engineering Leaders via stable counterparts.
Every Software Engineer in Test (SET) takes part in building our product as a DRI in GitLab’s Product Quad DRIs. They work alongside Development, Product, and UX in the Product Development Workflow. As stable counterparts, SETs should be considered critical members of the core team between Product Designers, Engineering Managers and Product Managers.
Every Engineering Manager (EM) is aligned with an Engineering Director in the Development Department. They work at a higher level and align cross-team efforts which maps to a Development Department section. The area a QEM is responsible for is defined in the Product Stages and Groups and part of their title.
Milestones (product releases) are one of our planning horizons, where prioritization is a collaboration between Product, Development, UX, and Quality. DRIs for prioritization are based on work type:
We use type labels to track: feature, maintenance, and bug issues and MRs. UX Leadership are active participants in influencing the prioritization of all three work types.
QEMs meet with their PM, EM, and UX counterparts to discuss the priorities for the upcoming milestone. The purpose of this is to ensure that everyone understands the requirements and to assess whether or not there is the capacity to complete all of the proposed issues.
For product groups with a SET counterpart, QEMs are encouraged to delegate bug prioritization to the SET as the bug subject matter expert for that group. In these situations, QEMs should provide guidance and oversight as needed by the SET and should still maintain broad awareness of bug prioritization for these delegated groups.
While we follow the product development timeline, it is recommended that you work with your counterparts to discuss upcoming issues in your group’s roadmap prior to them being marked as a deliverable for a particular milestone. There will be occasions where priorities shift and changes must be made to milestone deliverables. We should remain flexible and understanding of these situations, while doing our best to make sure these exceptions do not become the rule.
Section-level members of the quad are QEMs, Directors of Development, Directors of Product Management, and Product Design Managers aligned to the same section. These counterparts will review their work type trends on a monthly basis.
The Test Platform sub-department has two on-call rotations: pipeline triage (SET-led) and incident management (QEM-led). These are scheduled in advance to share the responsibilities of debugging pipeline failures and representing Quality in incident responses.
Every member in the Test Platform sub-department shares the responsibility of analyzing the daily QA tests against master and staging branches.
More details can be seen here
Every manager and director in the Test Platform sub-department shares the responsibility of monitoring new and existing incidents and responding or mitigating as appropriate. Incidents may require review of test coverage, test planning, or updated procedures, as examples of follow-up work which should be tracked by the DRI. More details can be seen here.
Please note: Any manager or director within Test Platform sub-department can step in to help in an urgent situation if the primary DRI is not available. Don’t hesitate to reach out in the Slack channel #test-platform.
We have compiled a number of tips and tricks we have found useful in day-to-day Test Platform related tasks.
For more information, please visit our tips and tricks page.
The Test Platform Sub-Department helps facilitate the quad-planning process. This is the participation of Product Management, Development, UX, and Quality which aims to bring test planning as a topic before the development of any feature.
For more information, please visit our quad planning page.
A borrow is used when a team member is shifted from one team to another temporarily or assists other teams part-time for an agreed-upon period of time. Currently, we do not have an SET embedded within every product group, hence for product groups with no SET counterpart, the following would be the process to request one:
Andy Hohenner
Brittany Wilkerson
Grant Young
Harsha Muralidhar
Jim Baumgardner
John McDonnell
Nailia Iskhakova
Nick Westbury
Nivetha Prabakaran
Richard Chong
Sanad Liaquat
Sofia Vistas
Vishal Patel~SET Borrow label.Please note that the borrow request might not guarantee 100% allocation to the requested product group. The temporary allocation will depend upon ongoing priorities.
The list of all SET borrow requests can be seen here.
Blocking tests have met stricter reliability criteria than other tests in our test suite. When a failure is seen in a blocking test, it’s less likely to be flakiness and more likely to be a true issue.
For more information, please visit our blocking tests page.
The Test Platform Sub-Department helps facilitate the risk mapping process. This requires the participation of Product Management, Development, UX, and the Quality team to develop a strategic approach to risk and mitigation planning.
For more information, please visit our risk mapping page.
The Test Platform Sub-Department helps facilitate the test planning process for all things related to Engineering work.
For more information, please visit our test engineering page.
If you need to debug a test failure, please visit our debugging QA pipeline test failures page.
The Test Platform Sub-Department follows a regulation processes for ensuring efficient and consistent management of projects with clear guidelines. For more information, please visit our project management page.
The Test Platform Sub-Department maintains ChatOps commands for Test Platform sub-department which provides quick access to various information on Slack. These commands can be run on any Slack channel that has the GitLab ChatOps bot such as the #test-platform and #chat-ops-test channels.
Commands that are currently available are:
| Command | Description |
|---|---|
/chatops run quality dri schedule |
Lists the current schedule for on-call rotation |
/chatops run quality dri report |
Show current and previous Test Platform pipeline triage reports |
/chatops run quality dri incidents |
Lists currently active and mitigated incidents |
For more information about these commands you can run:
/chatops run quality --help
For test automation changes, it is crucial that every change is reviewed by at least one Senior Software Engineer in Test in the Test Platform team.
We are currently setting best practices and standards for Page Objects and REST API clients. Thus the first priority is to have test automation related changes reviewed and approved by the team. For test automation only changes, the Test Platform Sub-Department alone is adequate to review and merge the changes.
We use Fibonacci Series for weights and limit the highest number to 8. The definitions are as below:
| Weight | Description |
|---|---|
| 1 - Trivial | Simple and quick changes (e.g. typo fix, test tag update, trivial documentation additions) |
| 2 - Small | Straight forward changes, no underlying dependencies needed. (e.g. new test that has existing factories or page objects) |
| 3 - Medium | Well understood changes with a few dependencies. Few surprises can be expected. (e.g. new test that needs to have new factories or page object / page components) |
| 5 - Large | A task that will require some investigation and research, in addition to the above weights (e.g. Tests that need framework level changes which can impact other parts of the test suite) |
| 8 - X-large | A very large task that will require much investigation and research. Pushing initiative level |
| 13 or more | Please break the work down further, we do not use weights higher than 8. |
The Executive Summary of all KPIs can be found here.
Test Platform owns and maintains the following:
We have compiled a list of learning resources that we’ve found useful for Software Engineer in Test and Engineering Manager growth.
For more information, please visit our learning resources page.
7104f09a)
