Validity and Assumptions

Issues of Validity

The principles of the RCT have been used among medical researchers since before the growth of qualitative research strategies in the 20^th century. Because of this, while qualitative research often feels the need to justify its theory and methods, RCT reports may simply rely on their status as the ‘gold standard’ of research methods.
A study protocol that ensures precise interventions (especially with socially complex interventions), samples that are representative of the population, and control of variables outside the intervention is required to obtain results that are reliable, valid, and generalizable.

Internal Validity

Also referred to as causality
Refers to the ability of investigators to determine that the result of the study was directly caused by the manipulation of the independent variable (treatment)
When well designed, the RCT demonstrates high internal validity, due to its ability to identify the cause and the effect, and control for confounding variables

External Validity

Also referred to as generalizability
Refers to the extent to which the results of the study may be applied outside the sample, and be expanded to the population as a whole
RCTs often considered to have lower external validity, this can be increased to some degree assuming a sturdy study design, control of biases, and an adequate sample size representative of the population

Assumptions

There is danger related to the uncritical assumption that because the RCT is near the top in the hierarchies of evidence they are necessarily the best option and the trial will be “good”.
RCTs have not always led to the production of reliable evidence for clinical effectiveness, especially if the research protocol was poorly designed

Bias

Is the deviation of results from the truth, resulting from systematic errors in research methodology? These may be eliminated or reduced during a properly constructed RCT.

Selection bias

Refer to fundamental differences between the participants assigned to each of the study treatment arms due to the way in which the participants were assigned

The treatment and control arms of the study should be as identical as possible and this is ensured by allowing distribution of participants between the treatment and control arms to be done by chance (randomization)

Example: Participants with worse health are placed in the control group to emphasize the benefits of the proposed treatment in the intervention group

Performance bias

A bias that is introduced during the treatment phase, when subjects in one group are systematically given different care than the others outside of the intervention under investigation.

This form of bias is controlled by blinding both investigators and participants.

Example: If the investigator is aware of which group is receiving the treatment, they may wish to spend extra time assessing and evaluating that sample group, this extra attention may have an unintended effect on the outcome measure.

Attrition bias

This bias is a result of participant withdrawal or exclusion from the study that is more predominant in one group versus the other.

When this is not considered in the study results it can impact the outcome results.
It is important to include data from withdrawn participants in the final results reporting.

Example: Side effects of an experimental drug causes dropout of participants randomized to the treatment group. If the initial results from these participants are not included, results may be biased in favour of the treatment.

Detection bias

Also referred to as information bias, this occurs when there are systematic differences in the way information is collected for the groups being studied.

This may occur more frequently in subjective data collection versus objective data collection.
This bias can be reduced by having outcome measures clearly defined and measurement tools standardized and reliable.

Example: In a study investigating the effects of a specific dressing, two investigators/ observers judge healing rate differently.

Google Sites

Report abuse