5/12/2021, 2:00 PM - 4:00 PM 

Virtual Conference, Prerecorded Session

Alisú Schoua-Glusberg, Research Support Services Inc. 

Daniela Glusberg, Research Support Services Inc. 

Chandana Vuyyuru, Feinberg School of Medicine, Northwestern University 

Evelyn Velazquez, Feinberg School of Medicine, Northwestern University 

Stacy Bailey, Feinberg School of Medicine, Northwestern University 

To minimize survey error, in survey design we aim to develop questions that are as simple and easy to understand as possible, while measuring what we intend to. The goal in item development for tests and assessments is related but, unlike survey questions, items have right and wrong answers, and difficulty is part of the design, not something to be avoided. In the survey industry we cognitively test survey questions and respondent materials to ensure they are interpreted by respondents as the question designers intended. For test items, cognitive interviewing is also a useful tool: it can help uncover if an item is interpreted differently not only by different individuals, but also across specific subgroups. Cognitive testing is also a key tool for assessing the translation of survey questions, as well as test items, into new languages. In analyzing cognitive interviewing findings of test items, analysts must determine if items are not being interpreted as intended because of the formulation of the item, the language used, or because of cultural differences in the target population, rather than the intended difficulty the question was designed to uncover. In our presentation, we will discuss issues around the cognitive testing of test items, and how it can diverge from cognitive testing of translations of survey questions. We will use examples from recent cognitive testing of the Spanish translation of items for a new digital health literacy tool. We will present interpretation issues we found in testing including translation issues, differing cultural interpretations, and participant errors in responses which do not represent item problems at all, as they are precisely what the items want to measure: participant’s level of health literacy. We will discuss how we separated comprehension issues due to translation and cultural context from comprehension issues related to health literacy