The calibration of complex computer codes using uncertainty quantification (UQ)
methods is a rich area of statistical methodological development. When applying
these techniques to simulators with spatial output, it is now standard to use principal
component decomposition to reduce the dimensions of the outputs in order to allow
Gaussian ...
The calibration of complex computer codes using uncertainty quantification (UQ)
methods is a rich area of statistical methodological development. When applying
these techniques to simulators with spatial output, it is now standard to use principal
component decomposition to reduce the dimensions of the outputs in order to allow
Gaussian process emulators to predict the output for calibration. We introduce the
‘terminal case’, in which the model cannot reproduce observations to within model
discrepancy, and for which standard calibration methods in UQ fail to give sensible
results. We show that even when there is no such issue with the model, the standard
decomposition on the outputs can and usually does lead to a terminal case analysis.
We present a simple test to allow a practitioner to establish whether their experiment
will result in a terminal case analysis, and a methodology for defining calibrationoptimal
bases that avoid this whenever it is not inevitable. We present the optimal
rotation algorithm for doing this, and demonstrate its efficacy for an idealised example
for which the usual principal component methods fail. We apply these ideas to the
CanAM4 model to demonstrate the terminal case issue arising for climate models.
We discuss climate model tuning and the estimation of model discrepancy within
this context, and show how the optimal rotation algorithm can be used in developing
practical climate model tuning tools.