Piloting your survey
When you feel your survey is complete, you’re ready to pilot it. This means sending it out to a small number of people to understand how respondents feel about it and whether they’re able to understand the questions as you hoped they would.
Piloting a survey is an important step for many reasons:
- You can learn if your respondents are interpreting your survey questions as you designed them.
- You can identify any bugs or logic issues.
- It saves you time, if you end up having to re-run the survey due one of the above issues.
- If you have a small population to sample from, you can’t afford to re-run a survey due to preventable errors.
When piloting a survey, you’re essentially running a qualitative research study on your own survey, which is interesting in itself! Since you’re looking for qualitative feedback, that means you don’t need a large sample size to run your pilot. It’s recommended to aim for 5-8 respondents in total, depending on the importance of findings, and the length and complexity of the survey.
There are two recommended ways to pilot a survey:
- Unmoderated - With an unmoderated pilot, you can use UserTesting.com to have qualified respondents take the survey, with instructions to think aloud while giving their own interpretation of what each survey question is asking of them. Each participant session is recorded, so you can watch the results at your own pace. This is the most time-efficient option.
- Moderated - With a moderated pilot, you essentially conduct 1:1 interviews with qualified respondents taking the survey, with instructions to think aloud while giving their own interpretation of what each survey question is asking of them. The nice thing about a moderated approach is that you’re present to ask follow up questions (example: ‘What do you think would be a better way to ask that question?’). This option takes more time to plan, schedule, and execute on, but it does allow you to gain further insight and clarification.
Upon completion of a pilot, you’ll probably need to make some iterations to the flow, wording, and/or options within your survey. Be sure to run a follow-up pilot to ensure your changes are fully understood; don’t assume they will be! Keep piloting until you find your pilot respondents all understand the questions, wording, and flow of the survey.
46417d02)
