Reinforcement Learning Identifies Age-Related Balance Strategy Shifts
Huiyi Wang, Jozsef Kovecses, Guillaume Durandau
McGill University, Montreal, Canada
McGill University, Montreal, Canada
Abstract
Falls are one of the leading causes of non-disease death and injury in the elderly, partly due to the loss of muscle mass in a musculoskeletal disorder named sarcopenia. Studying the impact of this muscle weakness on standing balance through direct human experimentation poses ethical dilemmas, involves high costs, and fails to fully capture the internal dynamics of the muscle. To address these limitations, we employ neuromusculoskeletal modeling to explore the impact of sarcopenia on balance. In this study, we introduce a novel full-body musculoskeletal model comprising both the torso and lower limbs, with 290 muscle actuators controlling 23 degrees of freedom and supporting varying levels of sarcopenia. Using reinforcement learning coupled with curriculum learning and muscle synergy representations, we trained an agent to perform standing balance on a backward-sliding plate and compared its behavior to human experiments. Our results demonstrate that, without pre-recorded experimental data, both healthy and sarcopenic agents can reproduce ankle and hip balancing strategies consistent with experimental findings. Furthermore, we show that as the degree of sarcopenia increases, the agent adapts its balancing strategy based on the platform’s acceleration.
A 290 muscle full-body MSK model
A new, fully developed musculoskeletal model is built from myolegs and myotorso. The upper body is composed of 210 muscles with 24 groups and the lower body with 80 actuators. The full body is open-sourced in the link above.
Methods Overview
Results
Our simulation results, in comparison to previous experimentation with Afschrift et al. 2016 and 2018, showed high similarity in terms of joint range, maximum torque and angle exerted, as well as the onset timing of balancing movements.
Healthy Model
Sarcopenia Model
Citations:
Please consider citing this article if you benefit from our open-source framework:
@article{Wang2025,
title = {Reinforcement Learning Identifies Age-Related Balance Strategy Shifts},
url = {http://dx.doi.org/10.1101/2025.06.05.656344},
DOI = {10.1101/2025.06.05.656344},
publisher = {Cold Spring Harbor Laboratory},
author = {Wang, Huiyi and Kovecses, Jozsef and Durandau, Guillaume},
year = {2025},
month = jun
}