Methods Matter: P-Hacking and Publication Bias in Causal Analysis in Economics
Brodeur, A; Cook, N; Heyes, A
Date: 30 November 2020
Journal
American Economic Review
Publisher
American Economic Association
Publisher DOI
Abstract
The credibility revolution in economics has promoted causal identification using randomized control trials (RCT), difference-indifferences (DID), instrumental variables (IV) and regression discontinuity design (RDD). Applying multiple approaches to over
21,000 hypothesis tests published in 25 leading economics journals
we find that ...
The credibility revolution in economics has promoted causal identification using randomized control trials (RCT), difference-indifferences (DID), instrumental variables (IV) and regression discontinuity design (RDD). Applying multiple approaches to over
21,000 hypothesis tests published in 25 leading economics journals
we find that the extent of p-hacking and publication bias varies
greatly by method. IV (and to a lesser extent DID) are particularly problematic. We find no evidence that: (1) Papers published
in the ‘Top 5’ journals are different to others; (2) The journal ‘revise and resubmit’ process mitigates the problem; (3) Things are
improving through time
Economics
Faculty of Environment, Science and Economy
Item views 0
Full item downloads 0