9. Designing Causal Evaluations

"Statistical results with a causal interpretation have a stronger effect on our thinking than non-causal information." - Daniel Kahneman

Lesson Prerequisites

This lesson assumes you have completed the Causal Inference, Sampling for Surveys, and Power Calculations lessons.

0. Intro to the lesson

This lesson ties together previous lessons on optimal designs for evaluations (Causal Inference, Power Calculations), and walks through how to implement those designs in Stata.

1. Simple random assignment

Simple random assignment is easy to execute, but may not be desirable if crossovers or spillovers are a problem, and may not be as efficient as stratified random assignment.

2. SRA in Stata

The user-written gsample command in Stata makes different types of random assignment straightforward to implement.

3. Comparing random assignments

It's a good idea to check that your randomization code produces the same random assignment each time that you run it. The cfout command makes it easy to compare two randomizations and check for discrepancies.

4. Balance tests

Balance tests are useful as a quick check for obvious bugs in your code, but be careful not to over-interpret the results: if you randomized treatment assignment, then any differences in baseline data between treatment arms are necessarily due to chance.

5. Stratified random assignment

If you have any baseline or administrative data on relevant covariates for individuals in your evaluation sample, then stratifying is a no-brainer.

6. Clustered random assignment

Clustered random assignment is often necessary due to program implementation and/or to minimize spillovers, but it greatly reduces statistical power.

Additional Resources

Banner photo: Radial graphs showing the relationship between temperature and mortality in London in the 1840s. The British Library. Accessed from https://www.visualcapitalist.com/wp-content/uploads/2018/04/cholera-share.jpg