How to Design (and Analyze) a Business Experiment

Abstract

The rise of experimental evaluations within organizations — or what economists refer to as field experiments — has the potential to transform organizational decision-making, providing fresh insight into areas ranging from product design to human resources to public policy. Companies that invest in randomized evaluations can gain a game-changing advantage. Yet while there has been a rapid growth in experiments, especially within tech companies, we’ve seen too many run incorrectly. Even when they’re set up properly, avoidable mistakes often happen during implementation. As a result, many organizations fail to receive the real benefits of the scientific method. This article lays out seven steps to ensure that your experiment delivers. These principles draw on the academic research on field experiments as well as our work with a variety of organizations ranging from Yelp to the UK government.