Evaluating Claims to Knowledge

Citizens in a democratic society are often required to read and interpret media reports of scientific research. Health and environment research reports are, for example, commonly portrayed in the media. Sometimes the research reports appear to contradict each other and sometimes the reports promote more uncertainty than certainty. Understanding the terminology and concepts for describing a research study is increasingly important for responsible citizenry. Listed below are some of the terms and concepts that will help you understand and critique media reports of research.

Types of Studies

correlational study—the connection or degree of agreement (e.g.,–0.3, +1.0) is sought between two variables, often without controlling for other variables; correlational studies often lead to cause-and-effect studies
cause-and-effect study—one variable is manipulated and all other variables, other than the responding variable, are controlled
control experiment—see cause-and-effect study
clinical trial—a controlled study involving people; usually a final-stage, double-blind study

Design Factors

term of study—the duration of the experiment e.g., observations over 5 s, 30 min, 3 mon or 15 a; long-term studies are most often preferred
sample size—the number of entities or people in a study; generally large sample sizes are preferred
random sample—one chosen randomly from the population of entities (to reduce bias)
replication—repetition of a study, generally, by an independent research group
placebo—in medicinal experiments, an inactive item (e.g., sugar pill) or treatment given to the control group
placebo effect—a beneficial effect arising from a patient’s expectations; present in both the control group and the experimental group
single blind—the subject (e.g., patient) does not know whether she/he has received the treatment or a placebo, but the experimenter knows
double blind—neither the subject (e.g., patient) nor the directly involved experimenter knows whether the subject has received the treatment or a placebo
control—a standard or comparison value, or procedure (e.g., leaving one of several identical samples unaltered for comparison), or a placebo
control group—a comparison group that does not receive the experimental treatment experimental group—a group that receives the experimental treatment
experimental group—a group that receives the experimental treatment

Nature of Evidence and Results

anecdotal—based upon personal experience or hearsay
reliable—reproducible or consistent time after time
valid—judged to be supported by adequate designs, materials, procedures, and skills
accurate—judged to be true or agreeing with the accepted value
precise—closely related or very similar; related to reproducibility of results
statistical bias—a sampling or testing error caused by systematically favouring some outcomes over others
random result—a result that could be expected on the basis of probability (e.g.,50% heads and tails when flipping a large number of coins)
coincidence—a result that is accidental and irrelevant to the study
significant difference—a difference that is greater than could be randomly expected when an experimental group and a control group are compared
certainty—the degree to which something is accepted by an individual or community (e.g., the evidence may have a high or low degree of certainty);measured by, for example, counting significant digits

Reporting Research

refereed journal—an academic journal for which research papers are sent to subject experts to determine whether the report is of sufficient quality to publish; also called peer-reviewed journal
funding agency—a report in a research paper of who funded the research. This may be important when a bias might be considered due to the self-interest of a funder.
abstract—a short summary describing the research processes and results

Page updated

Report abuse