Theories of Change

Solving a problem you do not understand would be pretty challenging. This revolves around setting the stage and making sure that you understand the suitable context around the situation you are trying to resolve. It is often not necessary to write this analysis out for the reader, as other sections will provide that context, but it is definitely a useful and needed step to ensure the quality of the ToC. 

Example: You are looking at designing a ToC to improve the welfare of farmed fish in India. Before you begin, it may be worth familiarizing yourself with how fish farming works in practice, what types of farms are most common (e.g., the average size of farms, location, etc.), what farming practices exist currently, among other questions. Understanding this will help you realize how exactly welfare could be improved, what types of incentives farmers face currently, and what work could be feasible under current farming practices. 

Example: Say you are working with an early founder looking to increase demand for vaccines, assuming that people don’t have accurate information about the value of vaccines. If in reality parents do have the right information and want their children to get vaccinated, but the clinic is always closed, an information campaign will ultimately not achieve anything. Always try to understand whether the problem you are tackling actually exists, and what drives it. 

Example: If you, as an organization, are aiming to hold teacher training to improve learning outcomes, it would be handy to check whether problems in learning are actually driven by teacher quality, or whether contextual factors like conflict and poverty are affecting school attendance driving learning losses. Things will never be driven by just one factor, but it's always important to make sure you have made efforts to understand the context and environment you are researching. 

A logic chain outlines how activities will ultimately lead to the final goal of an intervention or organization. Different resources recommend varying approaches – we think it’s worth working backward from the stated goals back to inputs, as this ensures that you are keeping the ToC narrowly focused on the ultimate impact an intervention or organization wants to achieve (this is sometimes called “backward mapping”; Center for Theory of Change, n.d.). 

1. 1. Identify Impact/goals/ultimate outcomes: These are the desired end result of the intervention or long-term change. Whereas outcomes could often be intermediate results you care about (e.g. improved levels of learning), the long-term impact might be even farther out (e.g. improved life outcomes because of those higher levels of learning in schools). Some organizations are fine with having an intermediate outcome (such as increased vaccination rates) as the impact statement. However, we prefer centering goals or impact as the consequential end result (vaccines in themselves are not valuable, but rather it is the protection they give and the improved health of their recipients that we care about). If the distinction between intrinsic and instrumental values is a bit confusing to you, we recommend checking out this resource. For most cases we will look at in this program, the end result or impact can be captured in health, wellbeing, or welfare effects. 

   2. Identify Outcomes: Outcomes are the “intended results of a program” (Innovations for Poverty Action, 2016, p.4). They are what needs to happen to achieve the intended goals – for example, these could be outcomes like improved literacy that can then lead to improved work opportunities, then more income, and finally increased well-being. An Innovations for Poverty Action (2016) brief describes outcomes in more detail: “The provision of outputs is under the control of an organization to some degree – they are certainly related to the effort and effectiveness of the activities implemented. But outputs set in motion a hypothesized series of changes that rely partly on the quality of program implementation and partly on whether the assumptions and theories underlying the program hold, as well as whether there are unanticipated changes in the program environment” (p.4). Ideally, outcomes should be  Specific, Achievable, Realistic, Measurable, and Time-bound (SMART) (UK Government Analysis Function, n.d.).

   3. Identify Outputs: Outputs are often the direct deliverables of a program or organization (such as products or services) (Innovations for Poverty Action, 2016). These are usually not valuable in and of themselves since they derive their value from how they translate to outcomes (e.g., a vaccine clinic is valuable because it can lead to vaccinations which are the outcomes, and thereafter improved health for recipients which is the impact). 

   4. Identify Inputs: These are the essential program elements to reach the deliverables described above. Organizations do lots of things that keep programs running, such as hiring, maintaining office space, and so forth. Usually, we forgo these essentials in a ToC (though they are definitely important), as we want to focus on the core inputs that are of most relevance to the identified deliverables.  

Draw the causal connections between the elements you have identified. 

These are the conditions that have to hold for a certain part of a causal chain to work out how you want – i.e., one step to actually translate into the next. This process is about going through the causal chain slowly and step by step and critically asking about how that chain could break down, how it would work, and what implicit assumptions you have made about how the world works (basically asking “What needs to be true for A to cause B?”). Sometimes this also leads you to discover that there is an intermediary step that needs to be put in the causal chain that you had previously overlooked. 

Externalities: Some of the steps in the causal chain might have consequences beyond the ones you have outlined in your program. It is worthwhile thinking about how steps in the ToC work might have unintended consequences beyond the positives you have listed and how the work might negatively affect other actors. 

As you build your ToC, you will have collected some evidence on critical elements. Once you have outlined all the assumptions above we can start collecting more evidence on each of these. In some research projects, you will have a separate section in which you are asked to review the evidence on the main critical aspects of the ToC (e.g., “Do bed nets reduce the incidence of malaria?”). In the ToC section, it's good practice to at least display the evidence or reasons you have for an assumption holding as true. 

Indicate your level of certainty about each assumption: After you have collected and put in the evidence for each of the assumptions you can then use that information to indicate how certain you are about each step in the ToC. It is important to outline the major uncertainties and risks (we use colour coding), which allows us to then identify which evidence we should prioritise to collect in our more formal evidence review. For example, we have high certainty from clinical studies that once a vaccination is administered, the likelihood of death decreases but we have very high uncertainty if our information campaign increases the uptake of immunisations. Communicating the level of certainty can be done by colour coding the arrows or boxes, and ideally, can be accompanied by explicit probability estimates. 

Virtually all interventions in international development, animal welfare, or many of the other topics you may cover as an applied researcher working in impact-focused work will involve some element of behavior change. This may be a purchase you may want a client to make, a disposition for a teacher to adopt, or a harmful activity you are trying to persuade someone to cease, to give a few examples. In some cases, such as work developing mass media programming, changes in behavior are essentially the only lever you have to pull. 

Due to this, we think it is useful to borrow heuristics, tools and approaches from behavioral science and the discipline of social and behavior change (SBC). There is a lot of advice out there on how to apply these insights, so we keep it fairly cursory in this section. 

Human behavior is complex, and behavioral science theories don’t always work. However, when using the theories and ideas probably reduces the chance of designing ineffective behavior-change interventions – “a growing body of evidence suggests that interventions developed with an explicit theoretical foundation or foundations are more effective than those lacking a theoretical base and that some strategies that combine multiple theories and concepts have larger effects” (Glanz & Bishop, 2010, p.400).

Theories can guide the search to understand why people do or do not practice a given behavior, help identify what information is needed to design an effective intervention strategy, and structure thinking on how to design a program so that it is successful. However, don’t expect that an intervention will work just because a theory predicts that it should – behavior is multifaceted, and context is key. 

At AIM, we think that the COM-B model of behavior is among the most useful for our purposes of applied research (Michie et al. 2011). The COM-B model says that behavior occurs as an interaction between three necessary conditions:

Diagram of COM-B with definitions (adapted from Michie et al., 2011, p.4)

The material for this section reflects on behavioral science as a tool for AIM, and introduces a few other theories beyond the COM-B model. 

The material for this section reflects on behavioral science as a tool for AIM, and introduces a few other theories beyond the COM-B model. 

Core materials

• Filip Murár´s internal presentation for the AIM research team (2023) (video, ~26 minutes)

Further materials

• EAST: Four simple ways to apply behavioral insights (Service et al, 2014) (executive summary) 

• The behavior change wheel: A new method for characterizing and designing behavior change interventions (Michie et al., 2011) 

When conducting applied research to evaluate or design an intervention, we aim to maximize the usefulness of a ToC for the implementers and evaluators. The core materials for this section narrow down on best practices when building or evaluating ToCs, but we note a few core tips to keep in mind below.

• Identify actors, locations, and context wherever possible. The clearest ToCs explain who is supposed to take the action in each intermediate step (e.g. instead of ‘children are brought to immunization camps’, specify  ‘parents bring children to immunization camps’). This makes the mechanics of the intervention clearer to outside readers and makes it easier to think about and prepare for how each step could go wrong (e.g., why might parents not bring their children to immunization camps?).

• Split elements appropriately. The best ToCs break out significant, independent elements. For instance, you may see an outcome statement that looks something like this: “Government introduces regulation and enforces it”. This would be a mistake since these are two very different steps that require some further digging and may play out differently (legislation can be introduced, but not enforced). Another typical mistake is to link up outcomes, for instance, “parents take their children to vaccination leading to increased vaccination rates” (a good tip is that whenever elements have words like “leading to” this usually means that things have been combined unnecessarily). 

• Avoid making an arrow jungle. ToCs can get messy really quickly, try to avoid having arrows going everywhere or having too many elements. If people can’t really discern what’s going on, a ToC won’t be useful. 

• Provide context around timelines. It can be helpful to outline how much time each step takes and how much time passes between each step. The more specific you can be the better. Sometimes ToCs are not displayed chronologically (and that’s OK), so this advice does not apply to all ToCs. 

• Ensure that your impact or goal statement reflects the actual fundamental ambitions of the intervention or program. 

• Avoid adding unnecessary elements, or elements that are not part of the primary means in which the program or intervention adds value. 

ToCs at AIM are usually quite narrowly focused and focus on the assumptions made. We make efforts to design good ToCs and have kept improving our approach because the program participants who take on ideas we research appreciate clarity and information around the presumed strategy of the intervention. Page 19 of this report provides an example of an exceptionally detailed ToC section in a deep report. Given the uncertain nature of the intervention and the potential for many paths to actually achieve outcomes, AIM researcher Filip Murár spent quite a bit of time being transparent about his understanding of the burden and barriers to change. He then did his best to limit the ToC to the significant elements for it (it still has a lot of elements, many more than an average ToC of ours) and clarified which portions of the ToC were the likeliest to achieve change. 

ToC can also be great for starting to think about a problem when done quickly. ToC’s done in a short amount of time, like the one below, will have a lot of  room for improvement. In our experience, they are worth doing anyways to help clarify goals, and structure research questions, among other things. 

Some questions to ask when evaluating a ToC

Does the author present the best possible ToC for addressing this topic? Has the author made a convincing case for this ToC over all other ToCs, such as a comparison to other possible ToCs?

Is the ToC sensible?

• Does the ToC provide a believable path to how the intervention can cause change and impact?

Does the author identify the main assumptions underlying the ToC?

• Check for missed assumptions

• Adopt a critical, skeptical view on the assumptions and stress test them.

• Are all key assumptions identified addressed in the rest of the report?

Does the ToC provide a reasonable medium between too much detail and too much abstraction?

• Are there main required outcomes, such as increased demand or other behavior changes, displayed in the ToC?

• Is the ToC confusing?

Does the author correctly categorize inputs, outputs, outcomes, and impact/goals?

• Follow these guidelines for correctly categorizing Toc components.Here.

If the intervention is a mainly behavioral one, has the author provided some assessment of the behavioral components of the intervention using a method such as COM-B?

• Follow these guidelines for correctly conducting a COM-B ToC (here)

• This is an exemplar COM-B ToC from AIM research (page 16), another one here.. page 1 (here and here)

Core materials

• Introduction to Theory of Change (MIT, 2021) (this resource is aimed at social entrepreneurs, but very clearly introduces the core components of a ToC, so we think it is useful, video,  ~6 minutes) 

• Syntax Structure of Outcome, Output and Activity Statements  (Global Affairs Canada, n.d.) 

• Theory of Change: Laying the Foundation for Right-Fit Data Collection (Innovations for Poverty Action, 2016)

• Theory of Change (Rogers, 2014)