Research

Abstract. 

What scientists choose not to publish can be as important as what they do publish. This study introduces a novel measure of the reporting of research results in economics, and investigates the causes behind under-reporting. Across over 5,048 pre-registered hypotheses from 317 studies on the American Economic Association RCT Trial Registry that were encoded, we first show that only 42% of hypotheses, on average across studies, have publicly available results 8 to 9 years after pre-registration. Around 64.5% of these publicly available findings are null results (i.e., not statistically significant), contrary to widespread concerns about the non-reporting of null results. An RCT was carried out to test three plausible barriers to full reporting, namely, a lack of awareness, engagement, or resources. An intervention in which our team engaged directly with the authors’ research and their under-reported findings increased the share of hypotheses with publicly available results by 6.2 percentage points (11% of the missing hypotheses), though most of this increase reflected results that already existed but were buried or previously observed; genuinely new estimates account for only 2.5 percentage points. Authors explained a further 18% of missing hypotheses as cases where the planned research never occurred; these explanations document why a result is absent rather than providing it, and although they do little for publication bias, they help limit research waste by creating a record of which planned studies were never carried out. Neither a light-touch “awareness” intervention nor a more expensive research-assistance intervention added to reporting, indicating that recovery required hands-on engagement. Together, the findings indicate that the majority of pre-registered hypotheses in economics remain unreported, with 42% still unaccounted for even after our most intensive arm; that under-reporting is more complex than a simple reluctance to report nulls; and that recovering missing results is difficult and labor-intensive rather than low-cost. We conclude that there is still need for stronger incentives and infrastructure to ensure that pre-registered scientific plans translate into publicly accessible evidence. Toward that end, this study provides prototypes of how standardized results reporting could be incorporated into economics.