Validity & Reliability

Specification

• Objectivity, reliability and validity (internal, predictive and ecological).

Validity refers to whether something measures what it intends to measure, whereas reliability is often used interchangeably with ‘consistency’; are results consistent when a study or test is repeated? A multitude of factors can affect validity, included extraneous or confounding variables (if they have not be carefully controlled for) or a poorly operationalised hypothesis.

Reliability can only be discerned through further testing; are the same results gained time and time again, even if the study is conducted by a different researcher? Observations (which can be very subjective) are often tested for inter-rater reliability), see below. That is, different observers records are checked for consistency of results. If a study has not put in place good controls it will usually be unreliable and hence not valid either.

Validity

Here are some of the main types of validity referred to in Psychology (there are more than outlined here).

Internal Validity: The IV in the study is what caused the DV and not other factors. Can be confident of cause and effect.

Ecological Validity: the extent to which the findings of a research study are able to be generalised to real-life settings.

Population Validity: The extent to which the findings can be generalised to other populations of people.

Historical (Temporal) Validity: Can the findings of the study be applied to different time periods? Do the findings depend on fluctuating cultural or social norms or even advancements in knowledge.

Predictive validity: is the extent to which a score on a scale or test predicts scores on some criterion measure. For example, the validity of a cognitive test for job performance is the correlation between test scores and, for example, supervisor performance ratings.

Reliability

Reliability is the degree to which an assessment tool produces stable and consistent results.

Inter-rater reliability: What is the degree of agreement among raters? It gives a score of how much consensus there is in the ratings given by judges.