The goal of survey design is to reduce errors. One type of error is measurement error, which can reduce the accuracy of study results (Boynton & Greenhalgh, 2004; Story & Tait, 2019).
What is validity?
Validity is the extent to which we are measuring what we intend to measure or what we believe we are measuring (Story & Tait, 2019).
Types (Story & Tait, 2019):
Face (how questions are interpreted by the public)
Content (how questions are interpreted by experts with appropriate knowledge of a content area)
Concurrent (how well questions conform to similar accepted measurement tools)
Convergent (how well questions perform relative to other questions aimed to measure the same construct)
Divergent/Discriminant (how well questions uniquely measure the construct of interest relative to questions aimed to measure distinct yet similar concepts)
Construct (how well questions measure what they intend to)
What is reliability?
Reliability is the extent to which we can reproduce the same results or responses over time and across individuals or groups.
Types (Story & Tait, 2019):
Test-retest (response stability over time); measured by having the same group take the survey twice at different times
Interobserver (how different respondents answer the same question)
Intraobserver (response stability over time within a single participant)
Alternate form (how differently worded questions evoke similar responses)
Internal consistency (how well similar questions vary together in a sample)
Do I need to measure validity and reliability?
If an instrument is well-established in the field and has been previously shown to be valid and reliable in a similar population, the need for measurement of validity and reliability is lower (Story & Tait, 2019).
Measuring validity
Face and content validity can only be assessed qualitatively, not statistically (Story & Tait, 2019). Construct validity is challenging to measure, but comparison with a gold standard that is known to measure the same construct can be done (Story & Tait, 2019).
Measuring reliability
Statistical procedures are appropriate for measuring most types of reliability (Story & Tait, 2019).