Simultaneous Learning of Contact and Continuous Dynamics

We automated the collection of over 500 tosses of a two-link articulated object using a Franka Panda 7-DOF robotic arm.  We tracked the poses of the 2 links using TagSLAM then converted the two poses to minimal coordinates via an optimization problem.

Anonymized public google drive link:  Available now!

We aim to do model building, or system identification, of an unknown system with which a robot is interacting.  A Franka Panda arm repeatedly lifts and tosses an unknown articulated object onto a flat table, and cameras track the object's configuration over time.  This data is grouped into current and next state pairs.  We formulate an optimization problem that determines the best explaining contact forces given the observed dynamics transitions.  Our loss function penalizes those contact forces' violation of realistic contact dynamics, given the current belief of the physical parameters like geometry, friction, and inertia.  This loss informs the update of the physical parameters in a data efficient manner.  Then the converged physical parameters can be combined with a contact solver / simulator of choice to perform forward dynamics predictions.

In the figure above, the explicit and implicit representations during inference (simulation) are in blue in the top row at left and right, respectively.  Explicit models are typically trained with prediction-based losses (bottom left), as can implicit approaches as in DiffSim (bottom middle).  Our alternative trains implicit representations with a violation-based loss (bottom right).

Our approach utilizes both:

• Implicit representations:  leverages existing contact simulators parameterized by physical quantities of geometry, friction, and inertia to implicitly encode forward dynamics models.

• Violation-based loss:  uses the true observed dynamics transition to inform what contact forces must have occurred, not requiring forward simulation during training time.

This is in contrast to:

• Differentiable Simulation (DiffSim):  Uses implicit representations with a prediction-based loss.

• Explicit models (End-to-end):  Uses explicit representations with a prediction-based loss.

We present four scenarios above, in order from left to right columns:

1. Articulated object with real data.

2. Cube with real data.

3. Asymmetric object in simulation with "vortex" augmented dynamics.

4. Articulated object in simulation with "gravity" impaired dynamics.

The three rows are the following metrics, from top to bottom (see our paper for more details on these calculations):

1. Average position error over a toss trajectory rollout started from an initial condition.

2. Average angular error over a toss trajectory rollout.

3. Volumetric geometry error.

Discussion

Both real experiments (articulated object and cube in left and left middle columns, respectively) show separation between CCN, DiffSim, and End-to-end methods, with CCN at least matching and often outperforming the alternatives along all metrics, especially with more training data.  On the cube dataset, CCN and CCN-R consistently converge to volume error of less than 10%.  DiffSim struggles to improve its volume error, even with more data.  On the real data, inclusion of a residual does not significantly help either the CCN or DiffSim approaches.  The residual makes improvements in the simulated examples, where CCN-R in the vortex scenario (right middle column) improves its positional trajectory error significantly with more data, achieving consistently 5x better performance than any other method by the largest tested dataset size. On the same metric, DiffSim-R sees no improvement beyond DiffSim.  In the gravity scenario (right column), the residual helps CCN-R maintain its good performance achieved at the correct gravitational acceleration, across all initial gravitational acceleration models.  In contrast, DiffSim-R outperforms DiffSim at every initial gravitational acceleration model.  Since this gravity scenario swept over different modeled gravitational accelerations, End-to-end is unaffected since its representation is unstructured.  Thus it is expected that its performance is consistent over the x-axis of these plots, and its results are included for reference against CCN and DiffSim.