Running Economics Experiments

Economic experiments in the lab and in the field allow for transparent and compelling causal estimation. MIT's Jameel Poverty Action Lab (JPAL) offers a five-day seminar that teaches the methods of randomization including choosing an appropriate sample size and how to address common threats to the experiment,

MIT Open Courseware provides these trainings in video and readings here and are available below.

It's unclear what becomes of experimental economics. Any monkey can run an experiment and the job market seems to punish students who invest heavily in experimental papers. And the method trades clear identification for clear external validity. Angus Deaton at Princeton is not a fan of economic experiments and explains his reasons here. Guido Imbens at Harvard disagrees and responds here.

Basic in running experiments is the difficulty of choosing an appropriate sample size. I have a useful tool here for doing so for almost every design. Don't mess to much with figuring everything out because it can take a lot of time. The tool here allows you to just input the relevant information about your data and experiment without getting waist-deep in power calculation formulas.

Topics covered: What is economic evaluation?

• Needs assessment

• Process evaluation

• Impact evaluation

• Economic cost-benefit analysis

Instructor: Rachel Glennerster

Topics covered: Why randomize in economics?

Instructor: Dan Levy

Topics covered: How to Randomize I

• Methods of randomization: Lottery, Phase in, Rotation, encouragement

• Multiple treatments

• Gathering support

Instructor: Dean Karlan, economist at Yale

Topics covered: How to Randomize II

• Unit of randomization

• Cross-cutting treatments

• Stratification

• Mechanics

Instructor: Rachel Glennerster

Topics covered: Measurement and Outcomes

• Key hypotheses

• Primary and intermediate outcomes

• Interpreting multiple outcomes

• Theory of change Model

• Questionnaire design

• Data collection/entry

Instructor: Esther Duflo, economist at MIT

Topics covered: Sample Size and Power Calculations

• Econometric estimation

• Hypothesis testing

• Power: significance level, variance of outcome, effect size

• Clustered design

Instructor: Ben Olken, economist at MIT

Topics covered: Managing threats to evaluation and data analysis

• Attrition

• Externalities (spillovers)

• Partial compliance and selection bias

Instructor: Michael Kremer, economist at Harvard

Topics covered: Analyzing Data

• Intention to treat (ITT) and Treatment on treated (ToT)

• Choice of outcomes and covariates

• External validity

• Cost-effectiveness

Instructor: Shawn Cole

Key Words: Economics, experiments, randomized-control trials, RCT, Andrew, Johnston, Andrew Johnston, economist, empirical, microeconomics, field research, lab research, experimental economics