Test and Tools Infrastructure Team

ℹ️ Note: This page is deprecated. The team has been restructured as Development Analytics under the Developer Experience Stage. Please visit the Development Analytics handbook page for current information.

Common Links

Category	Handle
GitLab Team Handle	@gl-quality/tp-test-tools-infrastructure
Slack Channel	#test-tools-infrastructure-team
Slack Handle	@test-tools-infrastructure
Team Boards	Team Capacity Board
Issue Tracker	quality/team-tasks

Vision

The Test & Tools Infrastructure team, under the Test Platform sub-department at GitLab, aspires to be a pioneering force in the innovation and maintenance of advanced testing frameworks and test infrastructure to support GitLab’s principle of Quality is everyone’s responsibility.

As an integral part of our vision, we aim to:

• Build a highly performant team of engineers, combining operational and software development experience to implement and support the best-in-class reliable testing processes & infrastructure.
• Work publicly in accordance with GitLab’s transparency value.
• Leverage dogfooding to prepare, build, deliver test tooling that improves the product itself.
• Align our strategy with the industry trends, company direction, and customer needs.

Mission

The key components of our mission are to:

• Make testing simple to enable easy contribution from everyone in the testing process, including test creation, execution, and debugging. This approach democratizes testing and fosters a culture of collective responsibility for quality.
• Make testing efficient to minimize time and resources while maximizing output, ensuring that testing is a facilitator, not a bottleneck, in the software development lifecycle.
• Make testing reliable to consistently deliver accurate and dependable results, ensuring trust in the test system & building confidence in every release.
• Foster innovation to explore, integrate, and build the latest technologies and methodologies in testing to keep GitLab at the forefront of software development innovation.

Team members

Name	Role
Abhinaba Ghosh	Engineering Manager, Test Platform, Test and Tools Infrastructure
Andrejs Cunskis	Senior Software Engineer in Test, Test and Tools Infrastructure
Chloe Liu	Staff Software Engineer in Test, Test and Tools Infrastructure
Dan Davison	Staff Software Engineer in Test, Test and Tools Infrastructure
David Dieulivol	Senior Backend Engineer, Development Analytics
Ievgen Chernikov	Senior Software Engineer in Test, Test and Tools Infrastructure, Analytics section
Jennifer Li	Senior Backend Engineer, Development Analytics
Mark Lapierre	Senior Software Engineer in Test, Test and Tools Infrastructure

Core Responsibilities

graph LR
    A[Test & Tools Infrastructure Team]

    A --> B[Test Infrastructure Optimization]
    B --> B1[Framework Enhancements<br>Upgrading testing frameworks to meet evolving needs]
    B --> B2[Infrastructure Efficiency<br>Optimizing test infrastructure utilization and cost]
    B --> B3[Reliability & Execution<br>Reducing test flakiness and improving execution speed]

    A --> C[AI-Powered Evaluation]
    C --> C1[Assessment Tools<br>Developing comprehensive tools for AI model evaluation]
    C --> C4[Generative & Predictive Test Analysis<br>Employing AI for enhanced testing accuracy]

    A --> D[Test Observability & Analytics]
    D --> D1[Metric Collection<br>Gathering comprehensive data and actionable insights on test execution]
    D --> D2[Performance Monitoring<br>Tracking key indicators of testing efficiency and effectiveness]

Team Roadmap

As part of our commitment to aligning with GitLab’s company goals, our team conducts a thorough review of company-level Roadmap and Objectives and Key Results (OKRs) at the beginning of each quarter. This process ensures that our efforts are strategically focused on delivering high-impact results that contribute to the broader organizational objectives. View the Test and Tools Infrastructure Roadmap for detailed insights and upcoming priorities

Metrics & Dashboards

The dashboards, owned by Test and Tools Infrastructure team, can be found in the Test Platform Dashboards.

How we work

• We prioritize asynchronous communication and a handbook-first approach, in line with GitLab’s all-remote, timezone-distributed structure.
• We emphasize the Maker’s Schedule, focusing on productive, uninterrupted work.
• Most mandatory recurring meetings are scheduled on Tuesdays and Thursdays.
• We reserve approximately 3-4 hours per week of focus time for learning new technologies and conducting research work, ensuring our team stays abreast of industry trends and fosters innovation in our projects. This time is dedicated to exploring emerging tools, deep-diving into subjects relevant to our work, engaging in self-directed learning, and experimenting with new ideas or proof-of-concepts. Team members have their calendars marked with this dedicated learning time. Any new meeting requests during these hours should be communicated beforehand, respecting this essential personal and professional development period.
• All meeting agendas could be found in the Team Shared Drive as well as in the meeting invite.

Work related rituals

Event	Cadence	Agenda
Standup notes	Once a week (Tuesday)	Share progress/blockers in #test-tools-infrastructure-team via Geekbot updates (
End-of-Week progress update	Once a week (Thursday)	Summarize status, progress, ETA, and areas needing support in the weekly update in each OKR issue. Examples: FY25-Q1 Test Platform OKR 2.2, FY25-Q1 TTI OKR 6
Team meetings and demos	Bi-weekly on Tuesday 2:30 pm UTC	Agenda
Monthly retrospective	Monthly	Contribute to retrospectives
Quarterly Business Report	Quarterly	Contribute to team’s success, learnings, innovations and improvement opportunities for each business quarter
Social hours	Quarterly	Virtual team building
1:1 with Engineering Manager	Weekly	Discuss development goals (see the 1:1 guidelines)

Project management

Feature work

• Each financial year, we develop a clear roadmap for the team to enhance visibility and alignment.
• Our roadmap preparation is an intensive month long exercise (usually last month of Q4), led by a DRI. During this phase, DRIs take the lead in drafting the roadmap using the roadmap prep-work template for necessary project tracks. This involves gathering inputs from the team, various stakeholders, assessing past performance, and aligning with the strategic goals of the Test and Tools Infrastructure Team.
• We utilize and plan OKRs to prioritize the roadmap items.
• Adhering to our team’s bi-weekly ritual, we conduct reviews to assess progress, address challenges, and recalibrate goals if necessary.
• We maintain a Team Board to visualise the current state of the feature work.

Working with us

We estimate ~20% of weekly time for support tasks, and balancing our roadmap work with emerging support needs. Please note this estimate varies depending upon ongoing priorities.

• For individual questions please reach out to the team via our slack channels: #test-platform, #test-tools-infrastructure-team.
• Raise support requests as issues in the team-tasks project. Add ~"team::Test and Tools Infrastructure" and ~"tti::support-request" labels.
• Ensure each issue is tagged with one of ~"type::feature", ~"type::bug, ~"type::maintenance following workflow classification guidelines.
• Team members analyze the issue and add a priority based on the Reliability Department’s prioritization framework. P1 issues will be taken up on an urgent basis the same week based on the availability of the relevant team members. Lower-priority issues are scheduled for review and discussion in our next bi-weekly meeting for capacity planning.
• The issues should be following the workflow label guidelines.

Borrow requests

Depending on the business needs, skillsets, timelines, and workload on SETs in other teams, members of the Test Tools and Infrastructure team could be requested to assist in other areas. In such case, the following process should be adhered to:

1. Initiate a borrow request issue with the ~SET Borrow label.
2. Submit requests at least two weeks in advance for adequate preparation and resource planning.
3. The leadership team of the Test Platform will triage and determine the appropriate team for each request. Once decided, they will signify this by adding a specific team label to the request. For Test and Tools Infrastructure team, the team label is ~“team::Test and Tools Infrastructure”.
4. Borrow requests are prioritized based on urgency and the potential impact on product development.

On-call activities

Test and Tools Infrastructure team follows the Test Platform on-call rotations: pipeline triage (SET-led) and incident management (QEM-led). These are scheduled in advance to share the responsibilities of debugging pipeline failures and representing Quality in incident responses. For more information, please visit our on-call rotation page.

Tooling

The team currently works cross-functionally, and our task ownership spans multiple tools and projects.

Tool	Description
gitlab/qa	GitLab’s Capybara-based end-to-end (E2E) testing framework
gitlab-qa orchestrator	An orchestration tool that enables running various end-to-end scenarios against any GitLab instance
GitLab Data Seeder	Data seeding harness that can seed customized data directly into a user or group namespace using low-level ORM calls
customers-dot/qa	E2E testing framework for The CustomersDot App
quality-pipeline-common	Reusable pipeline configurations for multiple testing projects within GitLab
dri	Tool that assists Pipeline Triage engineers to triage and report GitLab’s E2E pipeline test failures
chemlab	A Watir wrapper, a design scaffold and a CLI designed to join the existing testing practices at GitLab from two separate frameworks, the GitLab Capybara Framework, and the CustomersDot Watir framework into one
gitlab_quality-test_tooling	A Ruby gem for test-related GitLab automation/reporting
quality/toolbox	A collection of scripts for pipeline automation and analysis. Co-owned by the Engineering Productivity team
gitlab-qa-resources	Collection of terraform resources for Test Platform related infrastructure management. Co-owned by the Engineering Productivity team
gitlab-junior-toy-app	A toy app used for technical interviews while recruiting for the Software Engineer in Test (SET) position. Internal to the Test Platform sub department