Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics: Reply
Brodeur, A; Cook, N; Heyes, A
Date: 1 September 2022
Article
Journal
American Economic Review
Publisher
American Economic Association
Publisher DOI
Abstract
In Brodeur et al. (2020) we present evidence that IV (and to a lesser extent DID) articles are more p-hacked than RCT and RDD articles. We also find no evidence that: (i) articles published in the Top 5 journals are different; (ii) the “revise and resubmit” process mitigates the problem; (iii) things are improving through time. Kranz ...
In Brodeur et al. (2020) we present evidence that IV (and to a lesser extent DID) articles are more p-hacked than RCT and RDD articles. We also find no evidence that: (i) articles published in the Top 5 journals are different; (ii) the “revise and resubmit” process mitigates the problem; (iii) things are improving through time. Kranz and Pütz (2022) apply a novel adjustment to address rounding errors. They successfully replicate our results with the exception of our shakiest finding: after adjusting for rounding errors, bunching of test statistics for DID articles is now smaller around the 5% level (and coincidentally larger at the 10% level).
Economics
Faculty of Environment, Science and Economy
Item views 0
Full item downloads 0