Research Operations (ReOps) at GitLab

What is Research Operations?

Research Operations (often referred to as ReOps) is a discipline within Experience Research focused on managing and optimizing the processes, tools, and infrastructure that support research activities conducted by the Experience Research team. The primary goal of Research Operations is to enable Researchers to conduct their work more efficiently and effectively by providing the necessary support and resources.

Here are some key aspects of Research Operations in GitLab:

1. Participant Recruitment
2. Tool and Resource Management
3. Process Optimisation
4. Governance and Compliance
5. Training and support
6. Logistics and Administration

The ReOps team drives all of the above aspects. And often, Research DRIs can lead the ReOps aspects of their own projects, with the ReOps Specialist’s supervision.

Our Strategy

The current ReOps strategy is accessible internally only.

In general, we adhere to 3 pillars and 1 principle.

ReOps pillars

• Grow our reach, build and manage panels of niche participants meeting strategic research needs.
• Improve recruitment process efficiency and flexibility so Research DRIs can move fast with confidence.
• Drive the impact of ReOps and UX Research through communication, engagement, and community building.

ReOps principle

• Be transparent about our work and progress, and foster a culture of continuous improvement in our research process and practice, that everyone (UX Researchers, Product Designers, Product Managers, etc.) can participate.

How many studies can a Research Operations Specialist support?

The short answer is approximately 5-10 at any given time. The long answer is that it depends on the type of study and the target population. When each study only requires 5-8 participants who are relatively abundant in our database and community, the number of studies recruited per milestone could be ~10. This is also contingent upon the Product Designer and Product Manager submitting their complete requests early enough to allow specialists to juggle multiple recruiting efforts at once. When there are surveys that require 50+ respondents, or studies addressing new user groups or features, the number of studies recruited per milestone may be closer to 5. Those studies require sustained effort and creativity.

Never hesitate to bring up a recruitment request - the Research Operations Specialist will work with you to schedule the recruiting effort. Feel free to reach out to @Caitlin on Slack.

How does the Research Operations Specialist plan ahead?

When the number of requests exceeds 10, the specialist is at over capacity. When this occurs, there are too many projects to successfully manage at once, and the following side effects are experienced internally and externally to GitLab:

• Timelines slip
• Outbound communications from the Research Operations Specialist are reduced, resulting in the feeling of going dark
• Inbound communications to the Research Operations Specialist are increased, from people wanting to know statuses and help with next steps
• Errors occur in the process

To deal with situations of over capacity, we have two plans of action:

1) Quarterly planning

Given the nature of how we conduct research at GitLab, it’s difficult to have an accurate view into the pipeline at any given time to understand what research projects will be coming up when. However, we can rely on research project volume data from the past and predict the cadence of some projects quarterly to plan accordingly. An example of identified projects that require extra coordination are:

• SUS measuring
• Large N surveys (n=200+)

A quarterly planning session is conducted at the start of each quarter. The planning exercise maps out the upcoming research requests, by month, with known research projects, predicted research projects, upcoming events requiring research participants, anticipated changes to our processes (ex: new hires, unmoderated testing, etc), and specialist unavailability. In addition, check-ins at the beginning of each month take place between UX Research Operations Coordination and the UX Research Director.

The goal with the quarterly planning exercise is to have a fairly accurate view into the quarter at any given time, be able to identify any potential delivery risks, and to start early on addressing those risks with mitigation plans.

The specialist is responsible for driving and maintaining the research coordination quarterly planning exercise, the monthly check-ins, and keeping the Google Sheet up to date and accurate.

2) A back-up plan

To keep research projects on track, at least 2 other UX Researchers are trained to work as Research Operations Specialist when Research Operations Specialist are unavailable or if they are at over capacity and need additional assistance.

The Research Operations Specialist is responsible for:

• Developing training materials to train UX Researchers
• Conducting the training

Example coverage areas for training:

• Recrutiting intake process (ex: how to respond to all parties, Service Level Agreement (SLA), expectations of a specialist, etc)
• Communication guidelines when reaching out to panelists or recruitment sources (provide examples for each, special logins/accounts to use, etc)
• Handling gratuities for different countries
• Understanding the different types of issues created for a recruiting request vs. gratuity request - and wrapping those up

Ideally, a back-up plan is rarely used if proper planning has been done.

When over-capacity happens

Even with accurate planning, over-capacity can still occur. When this happens, specialists have several actions they can take:

• Inform the team - Be transparent. Let the #experience_research channel know, along with managers, that Research Operations Specialist is over capacity at the moment along with an estimated timeframe on when they expect things to get back to normal. This automatically sets expectations for existing and new requests. Lastly, follow-up with a post when capacity returns to normal to let the channel know.
• Reset expectations on any deliverable dates - Communicate to the key people on the projects impacted that their deliverable dates may change.
• Inform participants - Be transparent. If participants are negatively impacted, let them know that we’re busy, express a sincere apology, and reset timing expectations with them.
• Ask for help - Research Operations Specialists need to ask their trained UX Research Team back-ups for help. Research Operations Specialists will need to delegate the right duties to the UX Researchers to result in the biggest impact.
• New incoming requests will take longer - We don’t stop taking on new research when we’re at over capacity. Instead, we set expectations early with new requests. New requests will be buffered accordingly to accommodate being at over capacity and project owners are educated as to why.
• Encourage self-service - Research Operations Specialists can point people to self-serve resources with a link to instructions in the handbook (ex: using Respondent and User Interviews to find participants, etc).

When Research Operations Specialists are unavailable

Since Research Operations Specialists have unique duties that are critical to the success of research at GitLab, when they are unavailable, recruitment efforts slow down. This results in a series of problems for all parties.

To address this, two UX Researchers are available to act as back-up when specialists are unavailable. Back-ups complete any in-progress tasks, accept new requests, and keep existing requests on track.

Prior to the specialist becoming unavailable, they create an issue to provide their assigned back-up(s) with a clearly detailed list of:

• Research projects in process
• Actions needed for each project - and when to do them
• Key contacts for each project
• Links to all applicable issues
• Flag exactly what needs to be done by the back-up Research Operations Specialist

It’s the responsibility of the specialist to make sure their back-up understands their duties and responsibilities.

Can we combine recruiting requests to make recruiting go faster?

In some cases, absolutely. This works best when two or more studies are quite similar (e.g. both require interviews on a similar timeline and have an identical screener).

It is not usually advisable to combine recruitment for studies that are quite different from each other, for example a large survey and a set of user interviews. This is due to the different timeframes for these different studies.

In order to get people scheduled for interviews, we generally want to keep the feedback loop as tight as possible - by inviting qualified participants to schedule for an interview asap after they respond to the screener. This means that the whole recruitment process might take only a matter of days. Conversely, surveys requiring a large number of responses may require multiple rounds of emails through Qualtrics over a period of weeks. By the time you remember to reach out to some users who responded early on, so much time has passed that you’re likely to see a very low response rate. This results in us needing to recruit twice.

View page source - Edit this page - please contribute.