Research

Research interests

• Dynamics of Transformers. Analysis of mathematical models for Transformers, the main architecture of large-language models (LLMs) following the introduction of the self-attention mechanism. The relationship between inputs and outputs is understood from a systems perspective, leveraging techniques from dynamical systems on manifolds and control theory to investigate the behavior of Transformers.

• Geometric Mechanics and Field Theory. Description and analysis of physical systems using differential geometry and variational principles. I am particularly interested in the variational and Dirac formulations, which provide tools to model a wide range of situations, including nonholonomic systems, implicit systems, and interconnection.

• Reduction by Symmetries. Removal of superfluous degrees of freedom by exploiting the symmetry invariance of certain mechanical systems and field theories. The reduced differential equations are defined on a lower-dimensional space, simplifying the analysis and providing conserved quantities (Noether theorem).

• Discrete Mechanics and Field Theory. Discretization of continuous theories to construct geometric integrators for physical systems. Simulations based on these integrators exhibit significantly better long-term behavior compared to standard integrators due to the preservation of geometric structures.

Preprints

1. (with L. Colombo) Variational collision integrators for nonholonomic Lagrangian systems, 12 pp. (2025).

2. (with F. Gay–Balmaz and H. Yoshimura) Infinite-dimensional Lagrange–Dirac systems with boundary energy flow I: Foundations, 35 pp. (2025).

3. (with J.P. Silvestre and P. Tabuada) The asymptotic behavior of attention in transformers, 26 pp. (2024).

4. (with M. Castrillón López) Higher-order Euler–Poincaré field equations, 24 pp. (2023).

Journal papers

1. (with L. Colombo) Nonholonomic reduction for mechanical systems with collisions, SIAM J. Appl. Algebra. Geom., accepted, 26 pp. (2025).

2. (with R. Monjo and R. Campoamor–Stursberg) From colored gravity to electromagnetism, Gen. Relativ. Gravit., 56(117), 40 pp. (2024).

3. (with M. Castrillón López) Gauge reduction in covariant field theory, J. Phys. A: Math. Theor., 57, 42 pp. (2024).

4. (with M. Castrillón López) Higher order jet bundles of Lie group-valued functions, J. Lie Theory, 33(3), 831–844 (2023). 

5. Interconnection of Lagrange–Dirac systems through nonstandard interaction structures, Int. J. Geom. Methods Mod. Phys., 20(7), 28 pp. (2023).

6. (with M. Leok) Discrete Dirac reduction of implicit Lagrangian systems with abelian symmetry groups, J. Geom. Mech.,  15(1), 319–356  (2023).

7. (with M. Castrillón López) Principal bundles and connections modelled by Lie group bundles,  Geom. Dedicata, 217 (32), 26 pp. (2023).

Conference papers

1. (with J.P. Silvestre and P. Tabuada) Consensus is all you get: the role of attention in transformers, ICML 2025.

2. (with L. Colombo) Implicit nonholonomic mechanics with collisions, LHMNC 2024.

3. (with F. Gay–Balmaz and H. Yoshimura) Infinite dimensional Lagrange–Dirac mechanics with boundary conditions,  GSI 2023.

4. Covariant reduction by fiberwise actions in classical field theory, BYMAT 2020.

Other publications

1.  Reduction by fiberwise actions and higher-order reduction in Field Theory, Ph.D. thesis (2024).

2. Las simetrías de las ecuaciones del universo (in Spanish), Café y Teoremas, El País (2020).

3. (with N. Accomazzo, S. Baena, A. Becerra, J. Martínez, I. Soler and L. Roncal) Four different approaches to the fractional Laplacian, TEMat monográficos, 1, 47–60 (2020).

4. Reduction in geometric Field Theory: Yang–Mills equations , Master thesis (2019).