There are two types of validity: internal and external. Internal validity speaks to the rigorousness of the methods used to accurately and completely gather information from survey participants. External validity speaks to the usefulness of the survey beyond the study- its generalizability to various populations, contexts, etc. (Wiersma, 2011). Here we will explore the internal validity of survey studies.
As per Bhattacherjee, 2012 there are five biases that can invalidate a survey study. These biases are as follows.
Non-response bias: this bias is fairly self-explanatory. The response rate to surveys is generally low, regardless of if they are mail-in, in-person, or online. Bhattacherjee (2012) outlines some ways to combat this basis to improve the validity of a survey.
Advance notification
Relevance of content
Respondent-friendly questionnaire
Endorsement from a legitimate source corroborating the importance of the survey
Follow-up requests
Interviewer training
Incentives, both monetary and non-monetary
Ensuring the confidentiality and privacy of the participants
Sampling bias: occurs when some members of a population are systematically more likely to be selected in a sample than others. Limits the generalizability of findings because it is a threat to external validity, specifically population validity (Bhandari, 2022).
Social desirability bias: individuals typically want to be perceived in a positive light. Asking polarizing or tumultuous questions could encourage respondents to alter their responses to be viewed in a more socially desirable way. This will skew the results of the survey.
Recall bias: respondents may not adequately remember their own motivations or behaviors or perhaps their memory of such events may have evolved with time and are no longer retrievable. This could result in a response that is not reliable.
Common method bias: this occurs when the subject being investigated shares the same measurement design as the measured artifacts. One way in which Bhattacherjee (2012) suggests combatting this bias is to measure the independent and dependent variables at different points in time.
Survey studies rely on several different assumptions throughout their development, implementation, and dissemination. Here we will explore survey study assumptions as explored by Chelladurai (2020).
Standardization assumption: the assumption that all groups of people understand the survey questions in the same way- that all cultures will interpret the survey questions the way they were intended.
Respondent burden assumption: length of the survey, the time it takes to complete a survey, and the effort of the survey all contribute to respondent burden. Lessen the burden and respondents are more likely to partake in the survey.
Researcher assumption: the question may be based on researchers’ own experiences, biases, and contexts.
Reliability assumption: assumption that the research questionnaire and the answers provided by participants are reliable.