Gaussian process exploration-exploitation
By Eric Schulz (UCL London), Maarten Speekenbrink (UCL London) & Andreas Krause (ETH Zurich)
Abstract: Many problems in Cognitive Science require us to explore or exploit functions. Optimal design problems, model parameter estimation, as well as models of active human cognition all require us to model some trade-off between exploration (learning) and exploitation (maximizing) of an unknown function. Often times the parametric form of this function is unknown, hard to evaluate, or expensive to assess. In this talk we will describe how to use Gaussian Process regression, a non-parametric class of regression models, to explore and exploit functions. We will introduce and define Gaussian Processes as a distribution over functions used for non-parametric Bayesian regression and demonstrate different applications of Gaussian Processes. Examples will focus on pure exploration within optimal design problems, bandit-like exploration-exploitation scenarios including stimulus optimization and parameter fitting, and on scenarios with additional requirements such as safe explorations and the estimation of level sets.