Manipulated vs. measured: Using an experimental benchmark to investigate the performance of self-reported media exposure

Abstract

Media exposure is one of the most important concepts in the social sciences, and yet scholars have struggled with how to operationalize it for decades. Some researchers have focused on the effects of variously worded self-report measures. Others advocate the use of aggregate and/or behavioral data that does not rely on a person’s ability to accurately recall exposure. The present study introduces the prototype of an experimental design that can be used to improve measures of exposure. In particular, we show how an experimental benchmark can be employed to (1) compare actual (i.e., manipulated) and self-reported values of news exposure; (2) assess how closely the self-reported measures approximates the performance of “true” exposure in an empirical application, and (3) leverage the experimental benchmark to investigate whether a variation in question wording improves the accuracy of self-reported exposure measures.