Stochastic Optimization and its Application

Stochastic approximation provides a rigorous frameworks for designing, analyzing convergence, studying rate of convergence, etc. of the algorithms in stochastic optimizations, mini-batch computation algorithms, optimizations with sampling, etc. The classical results often require some continuity of the dynamics (equivalently, the continuous differentiability of the loss functions) and then characterize and analyze the convergence by dynamical systems generated from ODEs and the rate of convergence by diffusion generated from SDEs. However, these assumptions often fail in the modern practical problems and thus, it is necessary to propose a general (but also rigorous) framework, novel methods, and new insights for stochastic approximations with the discontinuous dynamics and/or the appearance of set-valued mappings, the non-smoothness of the loss functions, social (mean-field) interaction, and among others.

Related Publications: