1. Surrogate models and active learning
1.0. Gaussian process parameter estimation
Objective Bayesian Analysis of Spatially Correlated Data (2001).
Robust Gaussian stochastic process emulation (2018) Packages: RobustGaSP, RobustGaSP-in-matlab, PyRobustGaSP.
1.1. Surrogate models for high-dimensional dimensional input and output space
1.1.1. Gaussian processes for high-dimensional inputs or outputs
Bayesian emulation of complex multi-output and dynamic computer models (2010)
Parallel partial Gaussian process emulation for computer models with massive output (2016)
Gaussian process regression for materials and molecules (2021)
1.1.2. Neural networks for high dimensional inputs or outputs
Fourier Neural Operator for Parametric Partial Differential Equations (2023) demo, code.
Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs (2023)
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators (2021)
When and why PINNs fail to train: A neural tangent kernel perspective (2022)
1.1.3 Transformers
Attention Is All You Need (2017)
Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection (2024)
1.2. Surrogate models for discrete inputs
A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors (2019) LVGP R package.
Bayesian Optimization for Materials Design With Mixed Quantitative and Categorical Variables (2020)
1.4. Uncertainty quantification for the model without a probabilistic framework
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning (2016)
Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Network (2020)
A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification (2008)
A Tutorial on Conformal Prediction (2022) Another Note
1.5. Bayesian optimization and active learning
Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design (2010)
A Tutorial on Bayesian Optimization (2018). Packages: DiceOptim, GpyOpt , Bayesian-optimization.
Bayesian reaction optimization as a tool for chemical synthesis (2020)
Reliable emulation of complex functionals by active learning with error control (2022). R code.
Active Learning for Deep Gaussian Process Surrogates (2021). deepgp R package.
1.6. Connection between Gaussian processes and neural networks
2. Spatio-temporal models, image analysis and dynamical systems.
2.0. Kalman filter and its connections to
A New Approach to Linear Filtering and Prediction Problems (1960)
Kalman filtering and smoothing solutions to temporal Gaussian process regression models (2010)
2.1 .Latent factor model Kalman filter on spatio-temporal data
Fast data inversion for high-dimensional dynamical systems from noisy measurements (2024).
Generalized probabilistic principal component analysis of correlated data (2020). R code.
2.2. Dimensional reduction
Visualizing data using t-SNE (2008)
Dynamic mode decomposition of numerical and experimental data (2010)
On dynamic mode decomposition: Theory and applications (2014)
Active subspace methods in theory and practice: applications to Kriging surfaces (2013)
3. Data inversion and model calibration.
Computer Model Calibration Using High-Dimensional Output (2008)
Scaled Gaussian stochastic process for computer model calibration and prediction (2018). RobustCalibration Package.
A Theoretical Framework of the Scaled Gaussian Stochastic Process in Prediction and Calibration (2022).
Uncertainty quantification and estimation in differential dynamic microscopy (2021) DDM UQ MATLAB package
Ab initio uncertainty quantification in scattering analysis of microscopy (2023) AIUQ R package.
4. Scalable Gaussian processes and fast Bayesian computation.
4.0. Vecchia approximation
Vecchia Approximations of Gaussian-Process Predictions (2020). The GpGp package.
A general framework for Vecchia approximations of Gaussian processes (2021).
Sparse Cholesky Factorization by Kullback--Leibler Minimization (2021)
Scaled Vecchia Approximation for Fast Computer-Model Emulation (2022).
4.1. Inducing point approach
Variational Learning of Inducing Variables in Sparse Gaussian Processes (2009) SGPR in GPflow Python package
Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP) (2015) Gpytorch Python package
4.2. Posterior sampling
4.3. Bayesian numerical integration
Bayesian computation: a summary of the current state, and samples backwards and forwards (2015)
Probabilistic integration: a role in statistical computation? (2019)