Hard Contacts with Soft Gradients
Refining Differentiable Simulators for Learning and Control
Anselm Paulus*, A. René Geist*, Pierre Schumacher, Vit Musil, Georg Martius
Embedded Files
Hard Contacts with Soft Gradients
Abstract
Abstract
Contact forces pose a major challenge for gradient-based optimization of robot dynamics as they introduce jumps in the system's velocities. Penalty-based simulators, such as MuJoCo, simplify gradient computation by softening the contact forces. However, realistically simulating hard contacts requires very stiff contact settings, which leads to incorrect gradients when using automatic differentiation. On the other hand, using non-stiff settings strongly increases the sim-to-real gap. We analyze the contact computation of penalty-based simulators to identify the causes of gradient errors. Then, we propose DiffMJX, which combines adaptive integration with MuJoCo XLA, to notably improve gradient quality in the presence of hard contacts. Finally, we address a key limitation of contact gradients: they vanish when objects do not touch. To overcome this, we introduce Contacts From Distance (CFD), a mechanism that enables the simulator to generate informative contact gradients even before objects are in contact. To preserve physical realism, we apply CFD only in the backward pass using a straight-through trick, allowing us to compute useful gradients without modifying the forward simulation.
Contact forces pose a major challenge for gradient-based optimization of robot dynamics as they introduce jumps in the system's velocities. Penalty-based simulators, such as MuJoCo, simplify gradient computation by softening the contact forces. However, realistically simulating hard contacts requires very stiff contact settings, which leads to incorrect gradients when using automatic differentiation. On the other hand, using non-stiff settings strongly increases the sim-to-real gap. We analyze the contact computation of penalty-based simulators to identify the causes of gradient errors. Then, we propose DiffMJX, which combines adaptive integration with MuJoCo XLA, to notably improve gradient quality in the presence of hard contacts. Finally, we address a key limitation of contact gradients: they vanish when objects do not touch. To overcome this, we introduce Contacts From Distance (CFD), a mechanism that enables the simulator to generate informative contact gradients even before objects are in contact. To preserve physical realism, we apply CFD only in the backward pass using a straight-through trick, allowing us to compute useful gradients without modifying the forward simulation.
Top: Gradient-based MPC using CFD
Combining CFD with MJX enables gradient-based MPC of a musculo-skeletal robot using only a single distance between ball and goal as cost.
Top: DiffMJX and CFD add tuning knobs to improve gradients in MuJoCo XLA
DiffMJX improves the smoothness of the collision detector, gradient accuracy, and GPU memory consumption using Diffrax. CFD adds contact forces between non-colliding objects to the gradient computation while keeping the forward simulation untouched.
What is DiffMJX?
What is DiffMJX?
For hard contact settings or small simulation stepsizes, the gradients provided by MuJoCo XLA are erroneous. These errors stem from discontinuities in the system dynamics introduced by contacts, which affect the accuracy of numerical integration.
DiffMJX integrates Diffrax - an library providing numerical differential equation solvers - atop MuJoCo XLA.
In turn, DiffMJX adds the following functionalities to MuJoCo XLA:
Tradeoff simulation / gradient accuracy for compute time via adaptive stepsize controllers during integration.
Tradeoff GPU memory consumption for compile / compute time by using advanced checkpointing methods.
Switch between different methods for computing adjoints for backpropagation.
Left: Real-world cube parameter estimation
Estimation of cube parameters in MuJoCo via DiffMJX from real-world data of a cube being thrown onto a table. After training, the MuJoCo simulation accurately predicts the real-world trajectories. Despite contacts, the parameters are learned via run-of-the-mill gradient descent.
What is CFD?
What is CFD?
For using robot simulation in an optimization, we would like to have informative gradients. In particular, if two objects are not in contact, then the simulator gradients between the object states are zero. To obtain gradients between Contacts From a Distance (CFD), we extend MuJoCo's contact model to create small contact forces for positive signed-distances. This softened contact model is only used when computing simulator gradients keeping the forward simulation untouched.
Left: Illustration of CFD
During backpropagation the gradients of the forward simulation are replaced by the gradients of the soft forward simulation.
Comparisons of sampling-based MPC to gradient-based MPC with CFD
Ball catch
Switch ball positions in-hand
Bionic tennis
Code
Code
Coming soon.
🧋 🧑💻🍶
Page updated
Google Sites
Report abuse