Research prioritization
Though the UX Researchers are not the only ones who conduct user research in GitLab, we are responsible for ensuring all research are of high quality and impact-driven. As a small team, we apply the Global Prioritisation Framework and Process, so we can:
- Assure org-wide assessment and prioritisation of topmost research initiatives
- Create programs that break down silos and standardise data triangulation
- Reduce the risk of duplicating efforts (current and past work)
- Facilitate learning and collaboration among researchers
Who is responsible for research project prioritization?
UX Researchers at Gitlab work on multiple stage groups and often across stages. The UX Researchers provide a point of view on prioritization of the areas they support (See who’s supporting which areas). And ultimately, Group Product Managers or Directors of Product make the final decision on research project prioritization.
What is the cadence of UX Research global prioritization?
The UX Research team go through the process at the beginning of every quarter in a syncronised manner, aligning with the company’s OKR planning timeline. This is when the majority of research requests are raised, either by stakeholders or by UX Researchers themselves. These requests are then discussed, assessed and prioritized.
However, a good number of research requests are raised during the quarter. This is because things change, and priorities shift, especially in the fast moving tech industry. Ad-hoc research requests are discussed, assessed and prioritized using the same framework as the ones raised when the quarter starts.
When the team is at capacity, taking on new prjects requries stopping something else. New projects with higher priority bump lowest priority current work.
How the quarterly global prioritization works
Step 1. The In-Take
At the beginning of a new quarter, the UX Researchers check in with stakeholders in Product and Design on any changes and development in strategies / KRs / roadmaps and discuss possible new research projects or adjustments to existing ones. This can be done with a templated UXR Global Prioritsation Intake Issue.
Step 2. Understanding the Basics
Every research requests, either proposed by the stakeholders or by UX Researchers, goes through an initial and organic sense-checking phase, where we talk about the basics like problems to solve, research goals, what decisions the research will feed into, and the intended business impact type.
Every requests should have one main intended impact type clearly identified.
| Category | Impact types |
|---|---|
| Impact on Decisions | 1. Changes in strategy / Plan 2. Changes in product / design |
| Impact on Knowledge | 1. Knowledge gaps identified/filled 2. User’s perception of our product captured 3. Next big opportunity identified |
| Impact on UXR Maturity | 1. Improvement on UXR process efficiency 2. Improvement on UXR quality & reliability |
Some requests may be dropped at this phase as a result of people making connections and identifying existing insights, more efficient alternatives, or coming to the agreement that there is no clear case for business impact.
Some requests may be folded into other requests or existing projects due to overlapping goals, research questions and scope.
Step 3. Assessing Priority and Support Levels
All other requests are assessed and given a priority level from P1 to P4. This is done by the UX Researcher leading a discussion with stakeholders using our priority calculator as a guide (Demo here - internal access only).
Click to see more details
The priority calculator asks 6 questions and for each quetion gives a score from 1-5. The average score is mapped to the priority levels.
| Criteria | Question | Score/Weight |
|---|---|---|
| Impact | What is the business impact if we don’t do this research? | 1: Critical Impact - Major Risks 2: High Impact - Significant opportunity 3: Moderate impact - helpful insights 4: Low impact - nice to have 5: Minimal impact - no clear case |
| Users | How many users/customers would this affect? | 1: All users/critical segment 2: Large user segment 3: Medium user segment 4: Small user segment 5: Very few users |
| Dependency | Is this blocking any immediate decisions? | 1: Blocking critical launch 2: Blocking important decision 3: Input needed soon 4: Future planning 5: No specific decision |
| Timeline | What is the timeline flexibility? | 1: This quarter 2: Immediate (this month) 3: Within 6 months 4: Within 9 months 5: No timeline pressure |
| Alternatives | Are there alternative ways to get this information? | 1: No alternatives exist 2: Limited alternatives 3: Some alternatives 4: Some good alternatives 5: Many alternatives |
| Alignment | How does this align with our strategic goals? | 1: Core strategic priority 2: Aligns with quarterly goals 3: Support team objectives 4: Indirect alignment 5: No clear alignment |
| Priority Levels | Average Scores | Descriptions |
|---|---|---|
| P1 (Must Do) | <=1.5 | Critical business impact: Directly affects key company metrics/OKRs and yearlies; Blocking major product launch/release; Addressing unknowns about major product launch/release; Legal/compliance requirement High risk without research: Potential significant revenue loss; Major user experience degradation; Security/privacy concerns |
| P2 (Should Do) | <=2.4 | Strategic importance: Aligned with quarterly goals; Significant product decisions pending; Multiple teams/products affected. Clear business impact: Revenue opportunity > $X; Affects substantial user base; Time-sensitive opportunity |
| P3 (Nice to Do) | <=3.5 | Moderate business impact: Affects single product/feature; Medium-term decisions; Limited scope of impact Good opportunity: Improves existing experience; Supports team planning; Builds knowledge base |
| P4 (Backlog) | <=4.5 | Low immediate impact: Future planning; Non-critical improvements; Nice-to-have insights Can be delayed: No immediate decisions pending; Alternative data sources available; Lower strategic alignment |
| Will Not Do | >4.5 | Minimal business impact: No clear business case; Very limited scope; Duplicate of existing research Resource intensive: High effort, low return; Requires significant resources; Better addressed by other means |
We then use the support level calculator (demo - internal access only) to decide how UX Researcher should best support the effort.
Click to see more details
The calculator takes into account the following areas. Each criteria receives a score according to the table below and then is summed and divided by the total possible score to get a percentage which indicate the appropriate UX Research support level:
| Criteria | Description | Score/Weight |
|---|---|---|
| Issue | Link to research issue | N/A |
| Type | The type of research the project falls into: foundational, problem validation, solution validation. | Foundational = 3 Problem Validation = 2 Solution Validation = 1 |
| Ownership | Can this research be supported someone other than a UX Researcher? | Yes = 3 Somewhat = 2 No = 1 |
| Design Support | Is this project being requested by a Product team with Product Design support? | Not applicable for this research (study created/led by UX Research) = 3 Requesting Product team has Product Designers = 2 Requesting Product team does not have Product Designers = 1 |
| Complexity | Does this project involve multiple studies or methodologies? | Yes = 2 No = 1 I don’t know = 0 |
| Skill Development | Will this support skill development for the team or refine a process if a UX Researcher is involved? | Yes = 3 Somewhat = 2 No = 1 |
| Confidence | What level of confidence or knowledge do you have in the proposed solution or area of focus? | High = 3 Medium = 2 Low = 1 |
UXR Support Level is defined as the level of support the UX Researcher can commit to a given research project.
| UXR Support Level | Percentage |
|---|---|
| End to end (previously Gold) 🥇 | Greater than 80% |
| Task specific (preivously Silver) 🥈 | Between 51% and 80% |
| Consult/review (previously Bronze) 🥉 | 50% or less |
| UXR Support Level | Description |
|---|---|
| 🥇 End to end (previously Gold) |
DRI: UX Researcher What these projects look like: Large, strategic, rigorous projects that could benefit from a research specialist. Typically, foundational research, complex research questions, or high-priority problem validation. Who does what? The UX Researcher drives project management, aspects of execution, and completion of most tasks, but has support from Product and Design. While the UX Researcher is the DRI, the team is highly encouraged to participate in research sessions, analysis, discussions of results, and so on. Estimated number of studies: - 0.5 - 2 active projects (depending on UX Researcher’s level) Examples: - Research impacting multiple studies - Multi-method studies |
| 🥈 Task Specific (preivously Silver) |
DRI: Product/Design What these projects look like: These primarily consist of problem validation projects. Who does what? The UX Researcher takes on specified tasks within a study and advises on the rest. Product and Design drive project management aspects of execution and completing most tasks with support from the UX Researcher. Estimated number of studies: - 1 - 6 active projects (depending on UX Researcher’s level) Examples: - The UX Researcher and Product Manager or Product Designer collaborate on the research methodology, craft a script, or review an analysis. - The UX Researcher provides dedicated support for specific tasks that take less than a few days to execute. |
| 🥉 Consult/review (previously Bronze) |
DRI: Product/Design What these projects look like: These primarily consist of solution and problem validation projects. Who does what? The UX Researcher is consulted on specific aspects of a study. Product/Design is drives project management aspects of execution and completing most tasks, with advice from a UX Researcher. The team tags the UX Researcher in the issue to provide context and a due date for when feedback is needed. Estimated number of studies: - No more than 10% of the UX Researcher’s time should be dedicated to supporting these projects. Examples: - Reviewing an interview script - Participant recruiting criteria - Methodology choice |
Note that both calculators only generate recommendations. It is up to the UX Researchers and the stakeholders to decide and agree on the final priority level and the support level.
Step 4. Making and Communicating the Decisions
With the priority level and the support level agreed, the UX Researchers can estimate workload size in number of working days, and decide which projects to pick up, which ones to drop or push out - in agreement with their stakeholders.
To do that, we follow 3 team capacity rules:
- We keep 20% capacity for unexpected work (such as ad-hoc requests), training, and supporting each other (such as peer reviews and holiday coverage).
- We always take on P1 projects.
- When at capacity, taking on new priorities requres stopping something else. New higher priority projects bump lowest priority current work.
Once everyone is on the same page, the team communicate the projects picked up this quarter in relevant Slack channels, and Product and UX meetings.
Note that the current works are monitored and adjusted through out the quarter, in the following ways:
- Midway check-ins - a mini version of the global prioritisation process.
- Monthly backlog trimming
- Re-adjustment after receiving an ad-hoc request that needs to be picked up
How to handle ad-hoc requests
It’s normal to identify brand new research projects in an ad-hoc manner after prioritization happens, and we can still fold those new requests into the existing prioritized list. Here’s how to do that:
- The requester creates a research issue (template) and fills in the basic information.
- If the requester is a stakeholder in Product or Design, they assign the issue to themselves and the UX Researcher of their area. If the requester is a UX Researcher, they assign the issue to themselves and their Product and Design counterparts in relevant area.
- The stakeholders and the UX Researcher go though Step 2 - 4 as described above.
- If they decide to pick up the project, and the UX Researcher is at capacity, then they need to drop or off-load some current lower priority works, and communicate that with relevant stakeholders.
Note UXR leadership can be brought in to assist with the reprioritization.
Maintaining the research priorities list
Throughout the quarter, the UX Researchers will maintain the global research priorities list, keep the status of the projects up to date, and document any changes and adjustments in the Quarterly UX Research Global Priorities sheet (Example sheet - interal access only).
Soliciting feedback on the process
Optional: Towards the end of quarter, the UX Researcher will open a Retro comment thread to get feedback on how the process went for the team. This is an opportunity to reflect on the planning process, discuss how research went throughout that cycle, and identify any improvements to make in the next cycle.
Some questions you might consider in the retro comment thread:
- What went well?
- What did not go so well?
- Do you feel the most important research projects were prioritized and executed?
a4c83fb3)
