Analytic approaches

The quantitative science of model-selection has made concordant but independent strides, in both the physical and life sciences, toward assessing the ability of models to describe observations. Recent developments in particular assess both goodness-of-fit and (via an "Occam-factor") simplicity of presumption by using a log[1/probability] measure that Myron Tribus (we think) first referred to as "surprisal". Surprisal is measured using information units (like bits), and lies behind our understanding of thermodynamics as well as our methods for tracking genetic-relatedness and compressing-data, to mention only a few of its current applications.

Our focus here therefore is on log-probability or surprisal-based approaches to model-selection which focus on the multi-moment correlations between candidate models, and incoming observations of the world around. These April 2011 journal club talk slides may also provide some clues to their application in this context.