GenDOM: Generalizable One-shot Deformable Object Manipulation 

with Parameter-Aware Policy

So Kuroki, Jiaxian Guo, Tatsuya Matsushima, Takuya Okubo, Masato Kobayashi,

Yuya Ikeda, Ryosuke Takanami, Paul Yoo, Yutaka Matsuo, Yusuke Iwasawa

The University of Tokyo, Osaka University

ICRA 2024

ICRA2024_kuroki_video.mp4

Main result

Real2Sim2Real pipeline

Real2sim

Method - Gradient-based optimization

Deformable object parameters on simulation are estimated from real-world objects by taking loss from the grid density of point clouds between simulation and real-world via differentiable physics.  We can use the same setup for rope and cloth.

Qualitative evaluation

Rope 

The visualization of rope trajectories in the real world and simulation with optimized parameters from the three different release points, upper, middle, and lower.


                      optimized sim               real                         optimized sim               real                        optimized sim               real

Cloth

The visualization of cloth trajectories in the real world and the optimized simulation. 








Cotton


Ground truth

Optimized sim

  Point clouds (sim)








Rubber


Ground truth

Optimized sim

Point clouds (sim)

Abalation studies

Rope casting - The policy determines the momentary rope action so that the bottom end of the rope  reaches the goal coordinates.

The policy trained by both Young's modulus and Poisson's ratio (Upper) outperforms the policy trained without parameters  (lower).

The rope parameters and goal coordinates are the same in the column, but the results for actual reach coordinates are different.


                                                                                                                 Test1                                              Test2                                               Test3 

Sim2Real

Rope releasing - The policy determines the release coordinates so that the bottom end of the rope  reaches the goal coordinates.

Our method GenORM (upper), using only single demonstration, shows the almost same performance of πRD (lower) trained by 100 demonstrations. GenORM also outperforms other baselines. Please watch the demo video for more details.

Cloth spreading  -The policy determines the optimal acceleration for spreading the cloth over the table.

The policy trained by both Young's modulus and Poisson's ratio (Upper) outperforms the policy trained without parameters  (lower).

                                                                                                                           Towel                                                              Rubber

BibTex