We present a framework for Nesterov's accelerated gradient flows in probability space with applications in Bayesian inference. Here four examples of information metrics are considered, including Fisher-Rao metric, Wasserstein-2 metric, Kalman-Wasserstein metric and Stein metric.
For both Fisher-Rao and Wasserstein-2 metrics, we prove convergence properties of accelerated gradient flows.
In implementations, we propose sampling-efficient discrete-time algorithms for Wasserstein-2, Kalman-Wasserstein and Stein accelerated gradient flows with a restart technique.
We also formulate a kernel bandwidth selection method, which learns the gradient of logarithm of density from Brownian-motion samples.
Numerical experiments, including bimodal distribution sampling, Bayesian logistic regression and Bayesian neural network, show the strength of the proposed methods compared with state-of-the-art algorithms.
Yifei Wang, Wuchen Li, Accelerated Information Gradient flow, 2019.
The methods in this website are implemented in Matlab and Python (for Bayesian neural network) and available under the MIT license.
https://github.com/YiifeiWang/Accelerated-Information-Gradient-flow