UX Bash
A Bash is a research method where a group of participants, whether diverse or targeted, thoroughly use a feature to efficiently uncover as many pain-points as possible.
The idea is that each participant will use the feature differently, resulting in a large amount of varied feedback in a short amount of time. Bashes usually take place over 24 hours, enabling asynchronous participation.
Benefits of a UX Bash:
- Quickly and efficiently identify and prioritize usability issues.
- Discover software bugs, glitches, and inconsistencies.
- Uncover areas for feature enhancements and improvements.
- Collect user feedback to enhance the product experience.
- Turn around results to the team in 48 hours.
Bashes as an opportunity for early customer feedback
Bashes provide a valuable opportunity to gather early customer feedback by temporarily giving users access to features still under development. Using something like feature flags can make this process seamless for participants. For example, the /include feature was made available to external users during a bash via a feature flag and then turned off after the session concluded. This approach helps test features in a controlled environment and refine them based on real-world usage before broader release.
Note that UX Bashes are not meant to replace the team’s process to test features and identify bugs before release. The team should still test the software against their use cases and definition of done before doing a bash. Known problems should be addressed first, so that bash particpants can focus their feedback on less obvious problems.
When to iterate and run another Bash
UX bashes are most effective when used to uncover actionable insights and measure progress on identified issues. However, it is important to consider the timing of subsequent bashes.
During early bashes for Duo Chat, substantial feedback indicating that the speed of response was too slow. In such scenarios, it’s more effective to focus on addressing core issues before running another bash. Before scheduling a new bash, ensure that meaningful steps have been taken to address major feedback points from prior sessions.
Appropriate situations for a UX Bash
Bashes are useful when foundational research has already been conducted, but your team is trying to fine tune the experience before broader release, as quickly as possible. Bashes offer a cost-effective and efficient supplement to more extensive solution validation methodologies.
How to set up a UX Bash
Step 1. Define Bash Goals
- Clearly define what you want to achieve through the Bash (e.g., identify as many different bugs as possible).
- You can set specific goals for identifying usability issues, bugs, or feature enhancements (e.g., is the response time of an interaction fast enough).
Step 2. Create a Bash Guideline Issue and Bash Slack Channel
- A Bash Guideline Issue is designed to be the participant’s single source of truth for the Bash. It should include the objectives of the Bash, instructions on how to access the feature they are testing, instructions on how to deliver their feedback, and known problems that come up throughout the Bash.
- Create a dedicated private Slack channel for your Bash (ex. #duo_chat_bash_oct24). This allows participants to collaborate and seek clarification during the Bash. It is best practice to include the Bash Guideline Issue and your data collection mechanism as Bookmarks in the Slack channel.
Step 3. Determine Data Collection Approach
- Since the goal of the Bash is to collect as much feedback as possible, you need a way to collect that information. Qualtrics is one type of survey tool that allows multiple entries for a Bash, but you have the flexibility to select any feedback mechanism that works best for you and your team (i.e. GitLab issues, Google Forms, etc.). Your feedback mechanism should collect the information you and your team are most interested in, but should be short and to the point. We want to collect as much information as possible without becoming burdensome to the participant.
- For example, in the Duo Chat Bash, the team was most interested in participants’ perceptions of accuracy. Here is an example of some of the questions presented in a Qualtrics survey (Link to example survey):
- What was the question you asked Duo Chat?
- What was Duo’s response?
- Ratings of overall quality
- Ratings of the speed of the response
Step 4. Recruit
- Determine who will participate in the Bash. Ensure that participants accurately represent the target users of the feature in order to provide actionable feedback. You can choose to target a diverse group or a specific audience based on the goals of your Bash.
- Bashes don’t have a maximum number of participants, but can expand based on your needs, ability to recruit from a population, and your ability to analyze the data efficiently. Previous Bashes have usually consisted of 5-10 users.
For Recruiting Internal Participants
- In the #ux_research_volunteers Slack channel, make an official announcement about the upcoming Bash.
- Share that announcement to other Slack channels for additional exposure. If you are targeting specific audiences, you can strategically select which channels to share with.
- Examples of diverse channels: #random, #whats_happening_at_gitlab
- Examples of targeted channels: #security, #docs, #product, #development
- Create a calendar event as a placeholder and add people who sign up to the event
For Recruiting External Participants
- Join the GitLab Discord server, where the community and external users are active
- Reach out to Fatima Sarah Khalid (GitLab and Discord handle: sugaroverflow) to have your Discord profile noted as a GitLab employee.
- Create an Incentives Request issue to pay participants.
- Announce the Bash in the #announcements Discord channel with a link to a screener survey
- Select your participants.
- You can also work with Fatima Sarah Khalid (GitLab and Discord handle: sugaroverflow) to have your post shared on the GitLab Community Forum to boost awareness.
- Send event reminders
- Creating a calendar event for the Bash and emailing participants the day before the Bash is a best practice.
Step 5. Bash
- Remind participants about the Bash and its goals via the Bash Slack channel you created.
- Respond to questions as they come up during the Bash. Note: It is okay to not answer questions about technical limitations in order to get a closer approximation of natural behavior, but let the participant know that.
- Thank them for their participation at the end of the Bash.
Step 6. Reflect
- Analyze data. This will depend on the data you chose to collect, but this will usually involve categorizing qualitative feedback and summarizing quantitative ratings. (Example Bash Analysis Document)
- Share findings with your team. (Example Issue Update)
- Gather feedback from participants in order to improve subsequent Bashes.
a4c83fb3)
