How and when do pre-analysis plans, hypothesis registries, and replications help us know what we know and what we don’t know? A few simple models suggest the following take-aways: (i) Pre-analysis plans are most valuable for hypotheses that will not be tested multiple times and when the pre-analysis plans effectively eliminate researcher bias (not just reduce it), (ii) Pre-analysis plans can be valuable for replication efforts, and (iii) very few replications, typically three to five, are required to achieve precise, correct beliefs about the hypothesis, even with modestly biased replication attempts.
Journal of Political Economy, June 2012, 120(3): 359-397.
Intermediation Reduces Punishment (And Reward)
paper experimentally analyzes the schooling decisions of poor
households in urban Brazil. We elicit parents’ choices between monthly
government transfers conditional on their adolescent child attending
school and guaranteed, unconditional transfers of varying sizes. In the
baseline treatment, an overwhelming majority of parents prefer
conditional transfers to larger unconditional transfers. However, few
parents prefer conditional payments if they are offered text-message
notifications whenever their child misses school. These findings suggest
important intergenerational conflicts in these schooling decisions, a
lack of parental control and observability of school attendance, and an
additional rationale for conditional cash transfer programs: the
monitoring they provide.
(Web Appendix), (Data supplement)
American Economic Journal: Microeconomics, 3(November 2011): 77-106.
This paper shows moral decision-making is not well predicted by
the overall fairness of an act but rather by the fairness of the
consequences that follow directly. In laboratory experiments, third
party punishment for keeping money from a poorer player decreases
when an intermediary actor is included in the transaction. This
is true (i) for completely passive intermediaries, (ii) even though
intermediation decreases the payout of the poorest player and hurts
equity, and (iii) because intermediation distances the transgressor
from the outcome. A separate study shows rewards of charitable
giving decrease when the saliency of an intermediary is increased.
in The Oxford Handbook of Economic Conflict Resolution, 2012, Oxford Press, Rachel Croson & Gary Bolton eds.
"Can Subtle Provision of Social Information Affect What Job You Choose (and Keep)? Experimental Evidence from Teach For America" (Appendix)
Revise and Resubmit, American Economic Journal: Applied Economics
It has been well documented that information about the actions of others can affect small-stakes decisions. We show that a subtle provision of such social information can also influence a very high-stakes decision: whether to take (and keep) a job as a public school teacher. In an experiment involving thousands of admits to Teach For America (TFA), those provided with data about the high matriculation rate in the previous year are more likely to accept the job. Moreover, this effect persists into the second semester of teaching, even though one-sixth of those in the control group who initially accepted the job have left TFA by then. As expected, the effects are stronger for those more marginal in their decision to join TFA. Our results suggest that social information can have a powerful effect on high-stakes behavior and should be considered as a potential tool for policy.
Revise and Resubmit, Management Science
Measuring sexual orientation, behavior, and related opinions is difficult because responses are biased towards socially acceptable answers. We test whether measurements are biased even when responses are private and anonymous and use our results to identify sexuality-related norms and how they vary. We run an experiment on 2,516 U.S. participants. Participants were randomly assigned to either a “best practices method” that was computer-based and provides privacy and anonymity, or to a “veiled elicitation method” that further conceals individual responses. Answers in the veiled method preclude inference about any particular individual, but can be used to accurately estimate statistics about the population. Comparing the two methods shows sexuality-related questions receive biased responses even under current best practices, and, for many questions, the bias is substantial. The veiled method increased self-reports of non-heterosexual identity by 65% (p<0.05) and same-sex sexual experiences by 59% (p<0.01). The veiled method also increased the rates of anti-gay sentiment. Respondents were 67% more likely to express disapproval of an openly gay manager at work (p<0.01) and 71% more likely to say it is okay to discriminate against lesbian, gay, or bisexual individuals (p<0.01).The results show non-heterosexuality and anti-gay sentiment are substantially underestimated in existing surveys, and the privacy afforded by current best practices is not always sufficient to eliminate bias. Finally, our results identify two social norms: it is perceived as socially undesirable both to be open about being gay, and to be unaccepting of gay individuals.
"Pathways of Persuasion"
While economic theories of persuasion emphasize self-interest, others posit an important role for other-regard. For example, a salesperson might describe product features but also try to build rapport. We study these two mechanisms within a simple but rich experimental framework in which sellers, in a free-form conversation, try to convince buyers to raise their valuations for objects. We find that sellers benefit from communication despite their material conflict of interest. Communication affects both buyers’ self-interest and their other-regard. Changes in other-regard are mean zero, but interestingly a minority of sellers target other-regard and substantially outperform their peers. More generally, however, who is buying is actually a better predictor of persuasion than who is selling. Buyer-seller homophily also strongly predicts persuasion: gender-match, for example, more than doubles the sellers’ expected gain.
“Beliefs, Information and the Education Plans of Middle School Children in the Dominican Republic”
funded by research grants from the International Growth Centre, $1.26M grant from US AID, and Fundacion INICIA
In this paper we present the results of an evaluation of a large scale informative and persuasive social marketing campaign. This policy implementation and evaluation was conducted in spring 2015, in conjunction with the government of the Dominican Republic and had the objective of lowering high school dropout rates. The evaluation includes 25% of all public schools with middle school students and provides information on the returns to education, the availability of financial aid and also contain non-informative persuasive content. The information was provided through a series of four 20 minute videos with a telenovela format that include infographic segments and are shown in school to students in 7th and 8th grade. We develop the videos so that we can differentiate a treatment that has no statistical facts at all but is otherwise identical to an informative series of videos that include infographic information on earnings. We use a baseline and follow-up survey covering 40,000 students to measure how self-reported beliefs change regarding expected returns for self and the population, beliefs regarding the feasibility of different options, and more generally how students’ educational plans change with the treatment. In addition we develop an interactive survey tablet application that explains the statistical concepts that are then asked in a repeated panel form as in Wiswall and Zafar (2015). This allows us to elicit detailed information about expectations and model the different mechanisms through which this policy affects outcomes. In particular we differentiate between salience of options, updating regarding the dispersion of earnings, updating regarding the conditional distribution of earnings given education and the change in feasibility or availability of options.
"A Model of Information Nudges"
with Clayton Featherstone, Wharton, and Judd Kessler, Wharton
A growing empirical literature has demonstrated
that providing decision-makers with information (e.g. about the actions of
others or the returns to different actions) can affect behavior. However, the
literature lacks a theory that can explain when such interventions will have a
large effect or even the sign of the effect. We introduce such a theory, based
on simple Bayesian updating in a setting of binary choice. It yields the
following intuitive insight: the sign of the effect depends on whether the intervention
causes the marginal agent to update her belief up or down. Further, the
magnitude of the effect depends on both the density of agents at the margin and
how much those agents’ beliefs move when treated. We also suggest researchers
do not need to measure beliefs, but rather can proxy for the beliefs of the
marginal agent with the fraction taking the action in the uninformed group.
Utilizing this intuition, our model makes a strong prediction about how
treatment effect sign and magnitude will vary with the baseline proportion
taking the action in the control group. We perform a meta-analysis of
informational nudges in the literature and find that, even across very
different experimental settings, the sign and magnitude of the treatment effect
varies in exactly the way our theory predicts.
"Moral Perceptions of Advised Actions"
with Alexander Gotthard-Real
The standard rationale for, and measure of, consultants is the
information and insight they provide. We identify an additional role: Reducing
the punishment faced by those they advise. Through a series of experiments, we
show that selfishness is heavily punished, but that much of this punishment can
be avoided by hiring a consultant to advise selfish behavior. This is true
despite, by virtue of the design, the consultant is not an unbiased third
party: Through a relational contract incentive, consultants are motivated to
tell the principals what they want to hear. Further, the reduction in
punishment is not driven by information asymmetries: Not only does the
consultant not have any more information than the principal, punishment is
lessened whether or not the punisher knows with certainty the principal acted
selfishly, or whether she has to rely on the advise as a signal. The upshot of
these results is that, across our treatments, when consultants are available,
selfishness increases significantly.
"Intermediaries in Fundraising Inhibit Quality-Driven Charitable Donations"
Charitable donations are frequently raised by an intermediary: a fundraiser (that is not the charity) solicits and accepts donations and subsequently sends the proceeds to the charity -- e.g. a workplace campaign for United Way, a 5km walk for Susan G. Komen, or a cookie-selling campaign by a Girl Scout troop. Such fundraisers can greatly increase donations received by a given charity, but how do they affect what types of charities we support? This paper shows intermediary fundraisers can make donors insensitve to charity quality: Unattractive charities can receive the same financial support as an attractive charity. In a series of experiments, when donations are framed as going directly to the charity, attractive charities receive larger (between 68\% and 91\% larger average donation across studies) and more (between 19\% and 25\% higher likelihood of receiving a gift across studies) contributions relative to unattractive charities; however, when donations for the same charities are collected by (meaningless) intermediary fundraising campaigns, donations become statistically indistinguishable across charities. The fundraising campaign does not affect donor recall of charity identity or evaluation of charity quality; it simply precludes donors from using these data in the donation decision. Follow-up experiments suggest it is a superfluity of information in the intermediary fundraiser context that clouds the judgment of the donor.
“Expectations Do Not Determine Punishment”
This paper reports a series of laboratory experiments investigating the hypothesis that expectations affect punishment. Despite support from Moral Psychology and recent reference-dependence experiments in Economics, I find third party punishment does not respond to exogenous changes in expectations of the targeted party's behavior. I use a random process for revealing the true action taken by the actor. This process varies the expectation the punisher holds just before the truth is revealed. Expectations are shown to vary significantly and substantially. However, in non-parametric and instrumental variables regression analyses, expectations are shown not to affect punishment at all. This is true either when expectations are exceeded or failed.
Works in Progress
"Aggregating the Sands of Time: Small Consequence Decision-Making and Intertemporal Choice"
"Exact and Robust Replications: A Proposal for Replications"