11. Questionnaire Design
"To get answers, ask questions. To get good answers, ask good questions." - Israelmore Ayivor
Lesson Prerequisites
This lesson assumes that you have completed the Collecting High-Quality Data bootcamp lesson. Basic familiarity with primary data collection is not necessary but may be useful for recognizing how to apply the concepts in this lesson.
Intro to the Lesson
Thoughtful questionnaire design is an essential step in collecting high-quality data. In this lesson, we'll walk through key principles for designing a questionnaire to minimize measurement error along the survey response process. Additionally, you'll learn a number of tips for creating robust questionnaires through translating and piloting.
1. Deciding What to Measure
Deciding which outcomes you are going to measure is a necessary step before you can design your questionnaire.
2. The Survey Response Process: Comprehension
The first step of the survey response process is for the respondent to comprehend what the question is asking, and to comprehend it in the same way as other respondents and in the same way that you intended.
3. The Survey Response Process: Retrieval
The second step in the survey response process is for the respondent to successfully retrieve the requested information from their memory.
4. The Survey Response Process: Judgment & Estimation and Response Mapping
In the third and fourth steps in the survey response process, the respondent needs to draw inferences on retrieved information and fit the response to pre-specified options.
5. Mitigating Social Desirability Bias
When asked about sensitive subjects, respondents often try to adjust their response to fit socially acceptable norms. There are strategies that you can use in your question phrasing and formatting that can help ensure that you still get accurate data for sensitive questions.
6. Tips for Creating a Robust Questionnaire
We'll close out the lesson with key tips for creating a robust questionnaire, as well as best practices for translation and piloting.
Additional Resources
IDinsight repository of questionnaires
J-PAL resources on measurement & indicators
J-PAL resources on survey design
World Bank DIME Wiki on questionnaire design
Running Randomized Evaluations by Rachel Glennerster and Kudzai Takavarasha (book & website), especially Chapter 5 "Outcomes"
Banner photo: William Playfair's bar chart showing the price of wheat over time, an early example of a bar chart. Accessed from https://www.lindahall.org/william-playfair/