Research
Optimal Control of Large-Scale Multi-Angent Systems
Multi-Agent Systems (MAS) allow to model complex phenomena through a general paradigm combining endogenous interactions of the system together with external influences. Moving past the mere modeling purposes, a fascinating question is how to influence a system for inducing specific collective behaviours. The solution of this class of dynamical optimization problems is shadowed by the challenges deriving from the dimensionality of the system being controlled. This is usually described as the curse of dimensionality, referring to the exponential growth in storage and computational time required as the state space dimension increases.
In the MAS context, this pathological behaviour further worsen, as the dynamics describe the evolution of the system ensemble state, whose dimension scales according to both the number of agents being considered, and the space where the agent-to-agent dynamics take place.
The interesting double nature of the curse we aim at alleviating suggests a wide angle of possible techniques. This has motivated a variety of projects, all sharing the main goal of tackling down the computational challenges related to the (approximated) solution of OCP for many-agents systems. Here a taxonomy of the investigated techniques, organised according to the modeling resolution:
Microscopic
When operating at the level of agent-based control problems, the complexity of the solution is doubly cursed by the dimensionality of the system. In this context we propose methodology is to speed up readily available numerical solvers by means of gradient-augmented Neural Network approximation.
This allows to enhance the performance of the supervised learning task by including information about the gradient of the target variable. This additional information comes as a by-product of both the open-loop and closed loop solvers under consideration.
Gradient-augmented Supervised Learning of Optimal Feedback Laws Using State-dependent Riccati Equations, G. Albi, S. B. and D. Kalise. IEEE Control Systems Letters 6(2022): 836 -841 [arxiv, journal]
Data/moment-driven approaches for fast predictive control of collective dynamics, G. Albi, S. Bicego, M. Herty, Y. Huang, D. Kalise, C. Segala and D. Kalise [arxiv]
Macroscopic
Since the number of agents contributes to the system ensemble state dimension, when working with large systems a successful approach can be to model the dynamics in mean-field perspective, loosing the agent-based description in favour to the modeling of the agents density as a whole.
In this context we explore mean field optimal control for the FP equation, and an unsupervised Neural Network approach for solving mean field games for pedestrian dynamics in a room with moving obstacle.
Stabilizing control and deflation for the Fokker-Planck equation, S. B., D. Kalise, G. A. Pavliotis, [writing in progress, slide presentation]
Deep Galerkin Method for Pedestrian Mean Field Games, S.B, M. Butano, D. Kalise [github]
Mesoscopic
In many-agent systems characterized by high dimensional underlying dynamics, a mean field approximation could be not enough in terms of computational complexity reduction. A way of overcoming this is to treat the agents density as plasma matter, approaching the mean field solution from suboptimality by averaging the action of controlled sub-systems of agents.
Supervised learning for kinetic consensus control, G. Albi, S. B. and D. Kalise. Proceedings of the 25th International Symposium on Mathematical Theory of Networks and Systems (I), pp. 308-313 [arxiv, journal]
Control of high-dimensional collective dynamics by deep neural feedback laws and kinetic modelling, G. Albi, S. Bicego and D. Kalise [arxiv, slide presentation]
Collaborators
Dante Kalise
(PhD Supervisor)
Giacomo Albi
(M.Sc Supervisor)
Greg Pavliotis
Matteo Butano