Analytical perspective on appropriate dimensionality reductions in fully Bayesian inversions

Bayesian inversions supplement the information deficiency in naive model-parameter fittings by using the prior information on the physical phenomena. However, it entails an additional unknown as a relative weight (a hyperparameter) between the prior to the observation equation, which is often a tuning parameter questionable. Fully Bayesian inversion is a new paradigm to treat the hyperparameter as another random variable statistically evaluated from the data.

The fully Bayesian approach provides a (joint) posterior distribution containing all parameter information with uncertainty, but here is a less-known issue: the data interpretation highly depends on one's usage of the posterior. Sato, Fukahata & Nozue (2022) clarified the mathematical background of such dimensionality reduction problems and appropriate reduction, especially in the practically important linear inverse problem (a Gaussian model where data is a linear combination of the model parameters and noise and the hyperparameters are the variance scale factors of the likelihood and prior).

One surprise is the delta-functional concentration of the posterior, which is sometimes the central limit theorem but sometimes not. We find the marginal posterior of the model parameters and that of the hyperparameters respectively concentrate on 1) the mode of the joint posterior (MAP) and 2) the solution of Akaike's Bayesian information criterion (ABIC). The left figure compares MAP and ABIC estimates in volumetric strain rate inversions from GNSS data (N=572 components). Only ABIC provides appropriate estimates for small basis function intervals Δξ. We also found the cause that generally induces similar characteristics in high degrees of freedom (large M) models.

Full-Bayes approaches are now widely adopted with numerical schemes, and we solve their underlying issue of dimensionality reduction from an analytical perspective.

Asymptotic unbiasedness of ABIC proved

Now the more appropriate reduction is found to be a two-stage inference like ABIC, but in what sense is ABIC appropriate? Sato and Fukahata (submitted) proved 1) its mean convergence to the true value (asymptotic unbiasedness) in the model parameter estimation and 2) its mean convergence to the true value with zero variance (consistency) in the hyperparameter estimation for the linear inverse problem of large N and M. How appropriate ABIC is!