PLENARY TALKS:

Stefania Bellavia

TITLE: Stochastic Objective Function Free Algorithms for Optimization Problems

Abstract: This talk presents the class of Objective Function-Free Optimization (OFFO) algorithms—numerical optimization methods in which the objective function value is never computed, even though it is assumed to exist. First-order OFFO methods are widely used in fields such as machine learning and sparse optimization. Among these, the so-called adaptive gradient methods are particularly widespread (e.g., Adagrad, ADADELTA, Adam). All these methods share the characteristic of relying exclusively on current and past gradient information to adaptively determine the step size at each iteration.

Recently, it has been shown that Adagrad can be interpreted as a trust-region method where the radius is calculated without evaluating the objective function, making it significantly more robust to noise. This interpretation has paved the way for extensive research into both deterministic and stochastic OFFO methods for constrained and unconstrained problems. We present and discuss variants of Adagrad that are capable of utilizing (possibly very approximate) second-order information whenever available, can handle bound constraints on the variables, and can solve more general problems involving both equality and inequality constraints