9. Designing Causal Evaluations
"Statistical results with a causal interpretation have a stronger effect on our thinking than non-causal information." - Daniel Kahneman
Lesson Prerequisites
This lesson assumes you have completed the Causal Inference, Sampling for Surveys, and Power Calculations lessons.
0. Intro to the lesson
This lesson ties together previous lessons on optimal designs for evaluations (Causal Inference, Power Calculations), and walks through how to implement those designs in Stata.
1. Simple random assignment
Simple random assignment is easy to execute, but may not be desirable if crossovers or spillovers are a problem, and may not be as efficient as stratified random assignment.
2. SRA in Stata
The user-written gsample command in Stata makes different types of random assignment straightforward to implement.
3. Comparing random assignments
It's a good idea to check that your randomization code produces the same random assignment each time that you run it. The cfout command makes it easy to compare two randomizations and check for discrepancies.
4. Balance tests
Balance tests are useful as a quick check for obvious bugs in your code, but be careful not to over-interpret the results: if you randomized treatment assignment, then any differences in baseline data between treatment arms are necessarily due to chance.
5. Stratified random assignment
If you have any baseline or administrative data on relevant covariates for individuals in your evaluation sample, then stratifying is a no-brainer.
6. Clustered random assignment
Clustered random assignment is often necessary due to program implementation and/or to minimize spillovers, but it greatly reduces statistical power.
Additional Resources
Altman (1985) "Comparability of Randomised Groups" Journal of the Royal Statistical Society.
Athey and Imbens (2016) "The Econometrics of Randomized Experiments" Working paper.
Bruhn and McKenzie (2009) "In Pursuit of Balance: Randomization in Practice in Development Field Experiments" American Economic Journal: Applied Economics.
Duflo, Glennerster, and Kremer (2007) "Using Randomization in Development Economics Research: A Toolkit" CEPR Discussion Paper No. 6059.
McManus (2020) "Why you don't always need a baseline" and "When to collect baseline data" IDinsight blog.
Banner photo: Radial graphs showing the relationship between temperature and mortality in London in the 1840s. The British Library. Accessed from https://www.visualcapitalist.com/wp-content/uploads/2018/04/cholera-share.jpg