Inter-domain

Deep Gaussian Processes

Tim G. J. Rudner*, Dino Sejdinovic, Yarin Gal

University of Oxford



Inter-domain Deep GP with Doubly Stochastic Variational Inference.
(20 inducing points)

Conventional Deep GP with Doubly Stochastic Variational Inference.
(20 inducing points)

Abstract

Inter-domain Gaussian processes (GPs) allow for high flexibility and low computational cost when performing approximate inference in GP models. They are particularly suitable for modeling data exhibiting global structure but are limited to stationary covariance functions and thus fail to model non-stationary data effectively. We propose Inter-domain Deep Gaussian Processes, an extension of inter-domain shallow GPs that combines the advantages of inter-domain and deep Gaussian processes (DGPs), and demonstrate how to leverage existing approximate inference methods to perform simple and scalable approximate inference using inter-domain features in DGPs. We assess the performance of our method on a range of regression tasks and demonstrate that it outperforms inter-domain shallow GPs and conventional DGPs on challenging large-scale real-world datasets exhibiting both global structure as well as a high-degree of non-stationarity.

Higher Computational Efficiency & Better Predictive Performance

Top row: Standardized test RMSEs for varying numbers of inducing points (over ten random seeds).

Bottom row: Audio sub-band data over varying size and complexity.

Comparison of test log-likelihoods on datasets with global structure (over ten random seeds).

Inter-domain Deep GP with Doubly Stochastic Variational Inference.

Conventional Deep GP with Doubly Stochastic Variational Inference.