Characterising how neurons represent variables of the world in their activation has reshaped our understanding of the computations performed by biological and artificial neural networks. In parallel, there is mounting interest in understanding how dynamical systems perform computations on task variables through their internal dynamics. Here, we use concepts of Riemannian geometry to study the internal representations of dynamical systems. In particular, we showed that RNNs perform computations by dynamically warping their representation of task variables, for example compressing irrelevant information and expending relevant ones.

Pellegrino, A., & Chadwick, A. (2025). RNNs perform task computations by dynamically warping neural representations. Advances in Neural Information Processing Systems (NeurIPS) 38.

Recent work has argued that large-scale neural recordings are often well described by patterns of coactivation across neurons, which assumes that the data lies on a fixed-low-dimensional subspace. This view may overlook higher-dimensional structure, including stereotyped neural sequences or slowly evolving latent spaces. Here we argue that task-relevant variability in neural data can also cofluctuate over trials or time, defining distinct ‘covariability classes’ that may co-occur within the same dataset. To demix these covariability classes, we develop sliceTCA (slice tensor component analysis), a new unsupervised dimensionality reduction method for neural data tensors. 

Pellegrino, A., Stein, H., & Cayco-Gajic, N. A. (2024). Dimensionality reduction beyond neural subspaces with slice tensor component analysis. Nature Neuroscience, 27(6), 1199-1210.

Learning relies on coordinated synaptic changes in neural connectivity. Yet, previous methods to infer a dynamical system from data assumed the existence of a single fixed weight matrix. This discrepency made such methods fundamentally unsuited for data recorded from learning animals. To probe for such changes in connectivity and dynamics in data, we introduced low-tensor-rank RNNs which fits RNNs wiith connectivity connectivity.

Pellegrino, A., Cayco Gajic, N. A., & Chadwick, A. (2023). Low tensor rank learning of neural dynamics. Advances in Neural Information Processing Systems (NeurIPS), 36, 11674-11702.

Recent work has argued that machine learning models trained using gradient descent (or variants thereof) tend to have low-rank weights. But how this low rank structure unfolds over learning is unknown. To address this, we investigate the rank of the 3-tensor formed by stacking the weight matrices throughout learning. We derived several results bounding the rank of the gradient and weight tensor of linear dynamical systems and RNNs, using adjoint dynamics. 

Those results were published along ltrRNN in the appendix to:

Pellegrino, A., Cayco Gajic, N. A., & Chadwick, A. (2023). Low tensor rank learning of neural dynamics. Advances in Neural Information Processing Systems (NeurIPS), 36, 11674-11702.

The manifold hypothesis posists that low-dimensional manifolds can be ubiquitously found in high-dimensional data, and in biology such data are often generated by an underlying dynamical system. Yet, while dynamical systems models describe mechanistically the interactions between biological variables, manifolds are usually empirically extracted from data using dimensionality reduction methods. This has left elusive the link between biological geometry and dynamics. In this work, we argue that the low-dimensional manifolds seen in experimental data throughout the biological sciences are naturally born from dynamical systems principles. 

Pellegrino, A., Cornacchia, I., Cacyo-Gajic N.A., & Chadwick, A. (2025). Dynamical systems principles underlie the ubiquity of biological data manifolds. Manuscript in preparation.

The representation of task variables in the brain continuously changes even in the absence of learning, which is a phenomenon called "representational drift." Yet, neural network models trained under gradient descent in the absence of noise do not exhibit such an effect. Recent work has suggested that adding noise to the gradient dynamics is sufficient to introduce drift in the representation. Yet, in general such noise-driven drift can possibly change the intrinsic geometry of the representation, which is inconsistent with data. In this project, we investigate in artificial neural networks the conditions under which learning dynamics preserve intrinsic geometry of neural representation while changing the extrinsic geometry.

Pellegrino, A., & Palmigiano A. Ongoing project.