A. select/refine models

For uniform parameter-priors the most-likely parameters have maximum likelihood pmax[data|params+model] as well, and for log-normally distributed measurement errors this is nothing other than the least squares fit. A more challenging inverse problem is that of choosing models (or idea-sets) from a set of competing alternatives. The emergent cross-disciplinary strategy here is to choose the model whose probability p[data|model] marginalized over all possible parameter values is largest.

This amounts to choosing the model which is least surprised by the data i.e. for which cross-entropy Sp/r ≈ ln[1/p[data|model]] in nats is smallest. The marginalization process in effect adds an Occam-factor penalty[1] for fit-parameters to the minimized log-likelihood ln[1/pmax[data|params+model]], which can either be estimated by detailed analysis of parameter prior-probabilities or via information criteria like AIC (Akaike Information Criterion adds 1 nat/fit-parameter) now popular in the behavioral sciences[7] .