Single-point zeroth-order optimization (SZO) is useful in solving online black-box optimization and control problems in time-varying environments, as it queries the function value only once at each time step. However, the vanilla SZO method is known to suffer from a large estimation variance andslowconvergence, which seriously limits its practical application. In this work, we borrow the idea of high-pass and low-pass filters from extremum seeking control (continuous-time version of SZO) and develop a novel SZO method called HLF-SZO by integrating these filters. It turns out that the high-pass filter coincides with the residual feedback method, and the low-pass filter can be interpreted as the momentum method. As a result, the proposed HLF-SZO achieves a much smaller variance and much faster convergence than the vanilla SZO method, and empirically outperforms the residual-feedback SZO method, which are verified via extensive numerical experiments.

Quasi-stochastic approximation (QSA) is a deterministic analog of stochastic approximation, which is used to solve root finding problems commonly found in applications to optimization and reinforcement learning. Recent advances in QSA theory have led to algorithms with superior performance in terms of convergence rates and estimation error, when compared to their stochastic counterparts. In particular, a new representation of the solution to QSA algorithms provides a clear path to obtaining transient bounds and guidelines for algorithm design. New theory also brings methods for establishing stability of these algorithms. It was recently discovered that extremum seeking control (ESC) algorithms can be cast as QSA algorithms, and benefit from these advancements. In particular, recent QSA theory led to ESC designs for which global stability is guaranteed along with error bounds, provided the objective function is L-smooth. This talk will survey general theory, provide insights on how to design algorithms to improve performance, and illustrate design principles through numerical studies.

In this talk I will present a new constructive approach to extremum seeking (ES) by using a time-delay approach to averaging. Gradient-based ES of quadratic static maps via the classical and bounded ES algorithms will be considered. The practical stability conditions will be presented that are based on the transformation of the ES system to a time-delay one, where the delay is defined by the dither frequency. For the uncertain maps (with some known bounds on the extremum point and the Hessian), these conditions provide the first quantitative bounds on the frequency, the exponential decay rate and ultimate bound. For the unknown maps, new qualitative bounds are obtained. The results are extended to sampled-data and delayed implementations. The time-delay approach leads to quantitative bounds on the ES parameters for the ”grey box” models and explicit qualitative bounds for the ”black box” models making ES control reliable.

The first part of the talk will present an application context of Extremum Seeking (ES) in the field of quick localisation of victims buried by avalanche. The automatic search of avalanche victims relies on a professional search system called ARTVA. ARTVA is a system composed of two parts, an electromagnetic field generator and a sensor. On the one hand, the field generator, the so-called ”transmitter”, is held by the victim and is permanently active. On the contrary, the sensor, usually called the ”receiver”, is owned by the rescuer and activated only when the search phase is undergoing. The recent development of a 3D isomorphic receiver enlarged the spectrum of algorithms applicable to avalanche victims’ localisation, among which the ES represents an appealing option. In detail, the isomorphic antenna provides an attitude-uniform strength of the electromagnetic field, whose elaboration is demonstrated to be a proxy of the transmitter-receiver distance. This map represents the cost function the ES minimises. When applied to real scenarios, the performance of the ES is reduced by the environmental floor noise. This noise impacts the localisation in two ways: first, it makes the signal-to-noise ratio asymptotically vanish with the transmitter-receiver distance; second, it reduces the localisation accuracy. Regarding the ES tuning, these two effects lead to a design compromise. Indeed, on the one hand, the amplitude of the receiver movements should be sufficiently large to be robust to a small signal-to-noise ratio. On the other hand, the localisation accuracy is improved by smaller receiver movements. Moreover, stones and snow composition (layers of ice, fresh/compacted snow, air) impact the ideal electromagnetic field. In practice, the actual electromagnetic field leads to a non-convex cost function. Motivated by the applicative scenario presented before, the second part of the talk presents recent developments in the ES literature aiming to show ”uniform” properties of existing ES schemes.

In many applications, systems are required to operate in unknown environments. For example, high altitude pseudo satellites and underwater autonomous vehicles are required to operate in unknown environments. For such systems, the ability to control and regulate is primarily related to the ability to overcome the impact of the unknown environment. Extremum seeking control methods are ideal to handle such problems. In this presentation, we will present how extremum seeking control can achieve regulation to optimal conditions that are robust to the impact of exogenous variations.

In this talk, we present recent results for the design and analysis of equilibrium-seeking controllers (EqSC) with hybrid dynamics in the loop. These types of systems combine continuous and discrete-time dynamics during the seeking process, and their evolution in time is characterized by differential and difference inclusions rather than standard difference and differential equations. Examples of EqSC with hybrid dynamics include but are not limited to, purely continuous EqSC with optimizers described by set-valued mappings or non-Lipschitz dynamics with (practical) fixed-time stability properties, EqSC with arbitrarily fast and slow switching modes, EqSC with weakly-jumping parameters, EqSC with momentum and resets, distributed EqSCs for multi-agent systems with switching communication topologies, smooth EqSCs for switching plants, and EqSCs with hybrid filters and dither generators. Since solutions of systems described by hybrid and set-valued mappings are usually not unique, we do not insist on this property, but rather we characterize the behavior of all possible solutions generated by the closed-loop system. By exploiting recent advances in singular perturbation, averaging, and omega-limit set theory for hybrid dynamical systems, we establish practical stability results that parallel the results developed for ODEs. Some examples of common classes of seeking dynamics described by hybrid systems are also presented.

In this talk, I will discuss a recently introduced approach to design extremum seeking control for affine connection mechanical systems. The approach involves periodic perturbation signals with sufficiently large amplitudes and frequencies. Averaging results for mechanical systems under vibrational control reveal that the closed-loop system approximates the behavior of a certain averaged system. This in turn leads to the effect that stability properties of the averaged system carry over to the approximating closed-loop system. The averaged system is driven into the direction of symmetric products of vector fields from the closed-loop system. A suitable design of the feedback law ensures that the symmetric products point into descent (resp. ascent) directions of the objective function. Under suitable assumptions, one can show that minimum (resp. maximum) points of the objective function are asymptotically stable for the averaged system and therefore practically asymptotically stable for the closed-loop system. Several potential applications are discussed in the talk, such as source seeking control for nonholonomic agents and attitude control for satellites in space.

Extremum seeking (ES) is a powerful model-independent method for the control and optimization of unstable high-dimensional nonlinear and time-varying systems. In this talk I present an overview of recent ES applications for particle accelerators around the world which includes the coupling of ES with machine learning methods to make them more robust for time-varying systems. The topics covered include automatic beam control for an uncertain time-varying magnetic lattice at the SPEAR3 light source at SLAC National Accelerator Laboratory, for automatic beam loss minimization at the LANSCE proton linac at Los Alamos National Laboratory, for automatic longitudinal phase space control and pulse energy output maximization of free electron lasers (FEL) at the LCLS FEL at SLAC and at the European X-ray FEL at the German Electron Synchrotron  (DESY) research laboratory, for non-invasive 6D phase space diagnostics at the plasma wakefield accelerators FACET and FACET-II at SLAC, for online multiobjective optimization of electron beam trajectory and spot size at the AWAKE plasma wakefield experiment at CERN, and for beam control, optimization, and diagnostics at the HiRES ultra-fast electron diffraction compact accelerator at Lawrence Berkeley National Laboratory.

In this talk will discuss recent applications of extremum-seeking control (ESC) in industrial problems. From electromagnetic systems’ auto-tuning for elevators control, to real-time optimization of RF power amplifiers and airflow estimation for HVAC applications. We will review some of the main advantages of ESC compared to other auto-tuning techniques, as well as potential areas where theoretical advances are needed to match requirements from high-performance online computations in industrial applications. 

Exponentially-convergent extremum seeking is a significant result; Newton-based ES with a user-assignable exponential rate of convergence is even more valuable; and Poveda's time-invariant non-smooth fixed-time extremum seeking (FxT-ES) brings yet more benefit. Following this progression in providing the user with tools for for meeting an “optimization hard deadline” for systems with unknown models, I present results by my students Velimir Todorovski (FAU-Erlangen, TU-Munich) and Cemal Tugrul Yilmaz (UC San Diego) on extending time-varying model-based prescribed-time (PT) stabilization feedback to model-free prescribed-time extremum seeking (PT-ES). PT-ES is a time-varying alternative to the “fractional” time-invariant FxT-ES. While not sharing FxT-ES’s usability for persistent operation, PT-ES has been possible to extend to nonholonomic vehicles and PDEs. Todorovski’s result solves source seeking for mobile robots in GPS-denied environments. Yilmaz solves real-time optimization under large delays on the input and in the presence of PDE (partial differential equation) dynamics by building on PT stabilization results by Espitia and Steeves, extending to PT-ES the respective exponential ES results by Tiago Roux Oliveira presented in this workshop. PT-ES designs attain optimality in arbitrary user-prescribed time, independent (semi-globally) of the distance of the initial estimate from the optimizer.