Regret minimization in adaptive control

Trade-off in adaptive control

In many model-based adaptive control problems, it is vital to add an external excitation in order to guarantee the data informativity and/or an appropriate decrease in the uncertainties of the identified models due to the disturbances. Indeed, these uncertainties lead to control performance losses. However, this external excitation disturbs both the system output and the control effort which subsequently decreases the control performances. In both reinforcement learning and adaptive controller communities, significant effort has been spent in developing a framework in order to find an optimal trade-off between the performance degradation due to the uncertainties (exploitation cost) and the performance degradation due to the external excitation (exploration cost). It is called regret minimization, where the regret is a function of both the exploration and exploitation costs and the external excitation is designed in such a way that it minimizes the regret. It is an experiment design problem.

Related journal papers

Related conference papers