Regret minimization in adaptive control
Trade-off in adaptive control
In many model-based adaptive control problems, it is vital to add an external excitation in order to guarantee the data informativity and/or an appropriate decrease in the uncertainties of the identified models due to the disturbances. Indeed, these uncertainties lead to control performance losses. However, this external excitation disturbs both the system output and the control effort which subsequently decreases the control performances. In both reinforcement learning and adaptive controller communities, significant effort has been spent in developing a framework in order to find an optimal trade-off between the performance degradation due to the uncertainties (exploitation cost) and the performance degradation due to the external excitation (exploration cost). It is called regret minimization, where the regret is a function of both the exploration and exploitation costs and the external excitation is designed in such a way that it minimizes the regret. It is an experiment design problem.
Related journal papers
(Submitted) Kévin Colin , Håkan Hjalmarsson , Xavier Bombois. Finite-time regret minimization for Linear Quadratic Adaptive Controllers: an experiment design approach. Submitted to Automatica. Open access on arXiv: https://hal.archives-ouvertes.fr/hal-04360490. 2023
Kévin Colin, Mina Ferizbegovic, Håkan Hjalmarsson. Regret Minimization for Linear Quadratic Adaptive Controllers using Fisher Feedback Exploration. Joint publication in the journal IEEE Control System Letters and the proceedings of the conference IEEE CDC 2022, Cancún, Mexico. Open access on DiVA: https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1676885&dswid=6107. 2022
Related conference papers
Kévin Colin, Håkan Hjalmarsson, Xavier Bombois. Optimal exploration strategies for finite horizon regret minimization in some adaptive control problems. IFAC World Congress 2023, Yokohama, Japan. Open access on arXiv: https://arxiv.org/abs/2211.07949. 2023
Kévin Colin, Mina Ferizbegovic, Håkan Hjalmarsson. Regret Minimization for Linear Quadratic Adaptive Controllers using Fisher Feedback Exploration. Joint publication in the journal IEEE Control System Letters and the proceedings of the conference IEEE CDC 2022, Cancún, Mexico. Open access on DiVA: https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1676885&dswid=6107. 2022