Posterior belief assessment: Extracting meaningful subjective judgements from Bayesian analyses with complex statistical models
International Society for Bayesian Analysis (ISBA)
In this paper, we are concerned with attributing meaning to the results of a Bayesian analysis for a problem which is sufficiently complex that we are unable to assert a precise correspondence between the expert probabilistic judgements of the analyst and the particular forms chosen for the prior specification and the likelihood for the analysis. In order to do this, we propose performing a finite collection of additional Bayesian analyses under alternative collections of prior and likelihood modelling judgements that we may also view as representative of our prior knowledge and the problem structure, and use these to compute posterior belief assessments for key quantities of interest. We show that these assessments are closer to our true underlying beliefs than the original Bayesian analysis and use the temporal sure preference principle to establish a probabilistic relationship between our true posterior judgements, our posterior belief assessment and our original Bayesian analysis to make this precise. We exploit second order exchangeability in order to generalise our approach to situations where there are infinitely many alternative Bayesian analyses we might consider as informative for our true judgements so that the method remains tractable even in these cases. We argue that posterior belief assessment is a tractable and powerful alternative to robust Bayesian analysis. We describe a methodology for computing posterior belief assessments in even the most complex of statistical models and illustrate with an example of calibrating an expensive ocean model in order to quantify uncertainty about global mean temperature in the real ocean.
Danny Williamson is funded by an EPSRC fellowship grant number EP/K019112/1. We wish to thank an associate editor and two referees for their thoughtful and detailed comments regarding the paper. Their suggestions have helped to improve it a great deal. We also thank Adam Blaker and Bablu Sinha for running the NEMO ocean model to generate the ensemble that appears in our application.
This is the final version of the article. Available from International Society for Bayesian Analysis (ISBA) via the DOI in this record.
Vol. 10 (4), pp. 877 - 908