One World Seminar on Mathematics of Machine Learning

Abstract: The search for the optimal shape parameter for Radial Basis Function (RBF) kernel methods has been an outstanding research problem for decades. In this work, we establish a theoretical framework for this problem by leveraging a recently established theory on sharp direct, inverse and saturation statements for kernel based approximation. In particular, we link the search for the optimal shape parameter to superconvergence phenomena.

The analysis is carried out for finitely smooth Sobolev kernels, thereby covering large classes of radial kernels used in practice, including those emerging from current machine-learning methodologies.The results elucidate how approximation regimes, kernel regularity, and parameter choices interact, thereby clarifying a question that has remained unresolved for decades.

Abstract: The standard approach to generative modeling separates model fitting into two independent steps: first choosing a noise distribution to sample from, and then learning a transformation that maps these samples to the data distribution. In this work, we explore an alternative route that ties sampling and mapping. Our work draws inspiration from moment measures to explore a new factorization that links both sampling and action through a single convex potential, which we call the conjugate moment measure factorization. We propose an algorithm to learn the convex potential associated with this factorization in the generative modeling setting, where samples from the data distribution are available, and we validate this algorithm on generative tasks. In addition, we derive the Monge–Ampère equation associated with this factorization and propose an algorithm to learn the convex potential in a sampling context, when only the unnormalized probability density function is available.