Toward Provably Robust Deep Learning
Deep learning models achieve impressive performance across many domains, yet their internal mechanisms remain largely opaque. Learning is typically framed as a purely optimization-driven process, and trained models are treated as black boxes whose behavior is difficult to interpret, predict, or certify, particularly in nonstationary and continually evolving settings where formal guarantees are most needed.
The ultimate goal of this research program is to develop provably robust deep learning technologies, whose stability, adaptability, and failure modes can be characterized with mathematical rigor rather than inferred from empirical benchmarks. Reaching this goal requires a theoretical foundation that is still missing: a dynamical theory in which models are viewed as systems of fast-evolving internal states and slowly adapting parameters interacting over multiple time scales. From this viewpoint, learning, adaptation, and forgetting emerge from the joint dynamics of states and parameters, and tasks appear not as static objects but as dynamical configurations whose stability depends on time-scale structure, gradient propagation, and stochastic training effects.
Recurrent neural networks provide a natural starting point given their close connection with dynamical systems, but the underlying principles extend to modern architectures relying on gating, multiplicative modulation, and depth-induced time-scale separation. A longer-term ambition is to generalize the theory across the deep learning landscape by establishing a principled correspondence between temporal evolution and architectural depth. Depth can then be interpreted as a discretized time axis, allowing time-scale interaction, gradient transport, and learnability principles derived in recurrent settings to transfer to feedforward, convolutional, and transformer-based architectures. This time-depth equivalence provides a unifying framework toward learning systems whose robustness is not merely empirically observed but theoretically grounded and provably established.
References
L. Livi Learnability Window in Gated Recurrent Neural NetworksL. Livi. Time-Scale Coupling Between States and Parameters in Recurrent Neural NetworksP. Verzelli et al. Learn to Synchronize, Synchronize to Learn. ChaosP. Verzelli et al. Input-to-State Representation in linear reservoirs dynamics. IEEE-TNNLSA. Ceni, P. Ashwin, L. Livi, C. Postlethwaite. The Echo Index and multistability in input-driven recurrent neural networks. Physica DP. Verzelli, C. Alippi, L. Livi. Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere. Scientific ReportsA. Ceni, P. Ashwin, L. Livi. Interpreting RNN behaviour via excitable network attractors. Cognitive ComputationLearning on Graphs and Hypergraphs
Many natural and engineered complex systems (protein and metabolic networks, smart grids, brain networks) exhibit spatio-temporal interactions among many elements on multiple scales. Understanding such systems from data requires representations like sequences of graphs, and increasingly hypergraphs, that capture not only pairwise but also higher-order relations among interacting elements. Data-driven procedures for prediction, change detection, and control must therefore be designed to process spatio-temporal graph-structured data with formal robustness guarantees.
This direction extends the dynamical perspective of the previous research line to settings where time-scale interaction unfolds along two axes simultaneously: temporal evolution (equivalently, network depth) and spatial propagation across the graph. Within this unified framework, well-known pathologies of graph neural networks such as over-smoothing and over-squashing can be analyzed as manifestations of unfavorable time-scale coupling between depth and graph topology, rather than as isolated empirical failures. The same dynamical principles that yield provably robust deep learning in temporal settings can thus be reformulated to constrain how information and gradients propagate over both space and time on graphs and hypergraphs.
The ultimate goal is to develop robust control strategies for graph-structured learning systems whose stability, expressivity, and adaptability are certified jointly across temporal and topological scales. Learning on graphs and hypergraphs becomes a natural and demanding testbed for a theory of provably robust deep learning that must simultaneously handle multi-scale dynamics in time, depth, and space.
References
D. Zambon et al. Distance-Preserving Graph Embeddings from Random Neural Features. ICML 2020F. M. Bianchi et al. Graph Neural Networks with convolutional ARMA filters. IEEE-TPAMIF. M. Bianchi et al. Hierarchical Representation Learning in Graph Neural Networks with Node Decimation Pooling. IEEE-TNNLSD. Zambon et al. Change-Point Methods on a Sequence of Graphs. IEEE-TSPD. Zambon et al. Concept Drift and Anomaly Detection in Graph Streams. IEEE-TNNLSD. Grattarola et al. Learning Graph Embeddings on Constant-Curvature Manifolds for Change Detection in Graph Streams. IEEE-TNNLSD. Zambon et al. Autoregressive Models for Sequences of Graphs. IEEE-IJCNN 2019Prediction and Generation of Molecular, Material, and Particle Structures
Understanding the structure, dynamics, and properties of complex physical and chemical systems plays a fundamental role across the molecular sciences. Representative problems include the thermodynamics and kinetics of protein-ligand interactions in drug discovery, the prediction of stable phases and defect configurations in materials science, and the characterization of collective behaviour in interacting particle systems. In all such cases, the relevant objects (binding free-energy surfaces, potential energy landscapes, interaction networks) are high-dimensional and strongly nonlinear, and the quantities of practical interest, such as dissociation rates, phase transitions, and rare-event statistics, depend on special configurations like saddle points and transition states that are elusive to both simulations and experiments.
This direction develops applied methodologies for the prediction, characterization, and generation of such structures, building primarily on graph neural networks and generative models that operate naturally on the underlying topology of atoms, particles, and their interactions. Its distinctive feature is the integration of provably robust deep learning technologies developed in the other research lines: certified stability of graph-based representations, controlled generative procedures, and dynamical guarantees on training translate into formal reliability of the resulting tools. The aim is to deliver predictive and generative methods whose accuracy is not only empirically validated, but supported by the theoretical guarantees needed for scientific and industrial deployment.
References
Heydari et al. Transferring Chemical and Energetic Knowledge Between Molecular Systems with Machine Learning. Communications ChemistryGrattarola et al. Adversarial Autoencoders with Constant-Curvature Latent Manifolds. Applied Soft ComputingBrain and Physiological Signal Analysis for Health and Performance
Multichannel recordings from the human body, including intracranial and scalp EEG, electromyography (EMG), and other physiological signals, provide rich spatio-temporal data on the activity of neural and muscular systems. Such signals can be naturally represented as time-varying graphs over sensor networks or anatomical regions, exposing both the temporal dynamics of individual channels and the spatial structure of their interactions. This applied direction develops machine learning pipelines for the diagnostics, monitoring, and control of these systems, with two primary impact areas: clinical applications, where brain recordings inform the diagnosis and treatment of neurological conditions such as drug-resistant epilepsy, and sports science, where physiological data such as EMG support the assessment and optimization of athlete preparation.
References
Grattarola et al. Seizure localisation with attention-based graph neural networks. Expert Systems with ApplicationsLopes et al. Recurrence quantification analysis of dynamic brain networks. EJNDr. Cesare Alippi, Politecnico di Milano, Italy, and Universita' della Svizzera italiana, Switzerland
Dr. Robert Jenssen and Dr. Filippo Maria Bianchi, UiT the Arctic University of Norway
Dr. Peter Ashwin and Dr. Krasimira Tsaneva-Atanasova, University of Exeter, UK
Dr. Taufik Valiante and Dr. David Groppe, University of Toronto, Canada
Dr. Naoki Masuda, University at Buffalo, USA
Dr. Vittorio Limongelli, Universita' della Svizzera italiana, Switzerland
Dr. Stanislaw Drożdż and Dr. Pawel Oświȩcimka, Polish Academy of Sciences, Poland
Dr. Alessandro Giuliani, Istituto Superiore di Sanita', Italy
Dr. Witold Pedrycz, Universit of Alberta, Canada