Date: Wednsday, 5/10/2023 10:15 AM

Alisu Schoua-Glusberg

Research Support Services Inc.

The literature describes two different possible roles for cognitive testing interviewers. Willis and Artino (2013) and Miller et al. (2014) discuss how the role varies depending on the cognitive testing methodology. In think-alouds, the interviewer barely guides the respondent in expressing what they are thinking as they respond and may use standardized scripted probing administered to all respondents identically. With the probing technique, interviewers elicit a narrative from respondents that provides context from their lives to their answers to the survey questions, revealing how each question was interpreted and whether the response given matches that reality.

Unquestionably, most survey field interviewers can be trained on the first type, where probing is fully scripted and spontaneous probing not encouraged. Such approach may be best in projects with large numbers of interviews, in situations with no trained cognitive interviewers and limited training resources. The second approach yields richer results because there is no need to identify possible error a priori, thus reducing the effects of researcher bias. The point of this approach is to elicit patterns of interpretation and, through it, uncover response error wherever be present.

However, most studies cannot afford the time or cost of training survey interviewers as good qualitative interviewers. Interviewers must be able to understand testing goals for questions, how to probe on narrative to make sure the respondent’s reality supports their choice of answer, learn to ascertain when they have probed enough and when they need additional information. These are analyst tasks that have to be exercised on the fly during interviews. When writing interview notes, they need to distinguish what to include and what is not relevant.

This presentation will focus on the cognitive interviewer’s role as researcher and how to train survey field interviewers to be first line researcher-analysts using the probing technique.