4. Causal Inference
“Nothing’s random. Even if it looks that way, it’s just because you don’t know the causes.” - Johnny Rich
Lesson Prerequisites
Basic familiarity with impact evaluation and/or randomized controlled trials is useful but not necessary to complete this lesson.
0. Intro to the lesson
Causal inference is the science of estimating impact. In this lesson, you'll learn about some of the most common types of evaluation designs to estimate impact and the tradeoffs of each.
1. Defining impact & non-experimental designs
When we talk about estimating impact, we are trying to understand what would have happened in the absence of the program, or the counterfactual. In this lesson, we'll discuss some non-experimental approaches to approximating the counterfactual.
2. Randomized evaluations
A randomized evaluation (also known as a randomized controlled trial or RCT) is the most rigorous way to estimate impact. In this lesson, we'll discuss how RCTs eliminate selection bias and variants of the RCT design.
3. Quasi-experimental evaluations
Randomization isn't always possible, in which case we may attempt a quasi-experimental design, which require us to make more assumptions. Two common quasi-experimental designs that we'll discuss in this lesson are difference-in-differences and matching.
4. Threats to internal validity
Many challenges can crop up in an evaluation that will threaten the internal validity of your design. In this lesson, we'll learn about three common threats: non-compliance, attrition, and spillovers, as well as mitigation strategies.
Additional Resources
Running Randomized Evaluations by Rachel Glennerster and Kudzai Takavarasha (book and website)
J-PAL teaching resources on randomized evaluations and threats to internal validity
World Bank DIME Wiki on Experimental Methods
World Bank DIME Wiki on Quasi-Experimental Methods
World Bank Impact Evaluation in Practice
J-PAL chart on impact evaluation methods
Banner photo: Edmond Halley's map of trade winds, 1686. Accessed from https://commons.wikimedia.org/wiki/File:Edmond_Halley%27s_map_of_the_trade_winds,_1686.jpg