Evidence Reviews

Evidence reviews are an intrinsic part of virtually all research projects, whereby researchers seek to assess the relevant existing evidence pertaining to a research question. There is no catch-all answer to "good evidence,"  as this depends on the specific research question one is asking. However, there are conventions around the hierarchy of evidence for attributing cause-and-effect relationships. 

What is an evidence review?

An evidence review is a catch-all term for a somewhat structured process that reviews existing evidence in relation to a research question. As practitioners of impact-oriented research, evidence reviews are an intrinsic part of virtually all research projects we undertake, as we need to know what existing research can tell us about the decisions we face. 

Most of the time, we will be interested in understanding what the evidence for a specific causal relationship is. For example, when AIM researcher Vicky Cox looked into banning low-welfare animal product imports, she needed to assess whether a non-profit’s work was likely to cause a change in import policy (Cox, 2023). When AIM researcher James Che looked into participatory learning and action groups for maternal support, the main question he faced was whether putting these groups in place was likely to cause declines in maternal and neonatal mortality (Che, 2024). 

Evidence reviews of intervention effectiveness are closely tied to the ToC for the intervention itself, as you try to understand the logic and assumptions sustaining the intervention through the ToC and the evidence backing it. 

Other times, we will be looking at evidence to try to gain a descriptive understanding of a topic. This may involve gauging the evidence on the burden on a specific problem, the barriers leading to an issue, or the case studies for how a specific intervention works in practice. For instance, AIM researcher Morgan Fairless had to spend quite some time reviewing the evidence substantiating how big of a problem snakebites were, given the lack of consensus (Fairless, 2023). 

Evidence reviews are usually structured in steps:

1. Set a research question

2. Search for published evidence about the research question

3. Assess the studies collected

4. Draw conclusions based on the studies as evidence

What is “good evidence”?

The nightmare question that keeps researchers up at night. The best answer to this question is “it depends” – which can be a terribly useless (albeit often used) answer. However, we want you to take home the message that the quality of evidence depends – to a large extent – on the questions you are asking (e.g., do you want to learn how people feel about a certain disease plaguing a community? Perhaps a participatory research exercise may give you the richest detail on this; do you want to make a causal inference about the quality of a vaccine? There’s no question about it, a randomized trial or collection of studies will be your go-to). Quality of evidence is also relative and often practical, while randomized-controlled trials may be the “gold standard” for causal inference, they are often impractical to run, or cannot be run because of ethical concerns – sometimes it is straight-up impossible to randomize the treatment, and in those cases, a quasi-experimental approximation may be the best possible evidence for a particular question. 

Please remember this point on context and research questions as we review this section. You will note we will delve mostly into the quality of evidence for causal attribution, which does have a relatively straightforward hierarchy of quality. 

The hierarchy of evidence 

The hierarchy of evidence is a framework used in evidence-based medicine and social science research to assess the quality and reliability of different types of evidence for causal attribution. 

It organizes various study designs based on their methodological rigor, potential for bias, and ability to provide reliable answers to causal attribution research questions. The hierarchy allows researchers and healthcare professionals to determine the strength of evidence supporting a particular intervention or treatment. The hierarchy typically consists of several levels, with higher levels representing stronger evidence. 

Note that the hierarchy of evidence is a rough heuristic applied specifically to causal attribution. Different studies are better suited for specific research questions. You probably would not weigh a qualitative examination of focus group data very highly if you are trying to determine how a vaccine affects rates of disease, but you would probably value its insight in determining what feelings people have in relation to getting their children vaccinated. Additionally, it is good practice to avoid any brute applications of the hierarchy in terms of quantity of studies. One excellent RCT may be better than four badly designed ones, a quasi-experimental study may be more informative than a meta-analysis of observational studies, and so on. 

While the specific levels may vary slightly depending on the source or field of study, a common hierarchy includes the following:

Core materials

• Why is it so hard to know if you're helping? (Hoel, 2024) (~17 minutes)

•  How many people die from snakebites? (Dattani, 2023) (~12 minutes)

Click here for a breakdown of different methodologies and interpretation guidelines.  

Practice project and samples in our full PDF version.