So Kuroki, Jiaxian Guo, Tatsuya Matsushima, Takuya Okubo, Masato Kobayashi,
Yuya Ikeda, Ryosuke Takanami, Paul Yoo, Yutaka Matsuo, Yusuke Iwasawa
The University of Tokyo, Osaka University
ICRA 2024
arXiv -> https://arxiv.org/abs/2309.09051
ICRA2024_kuroki_video.mp4Main result
Real2Sim2Real pipeline
Real2sim
Method - Gradient-based optimization
Deformable object parameters on simulation are estimated from real-world objects by taking loss from the grid density of point clouds between simulation and real-world via differentiable physics. We can use the same setup for rope and cloth.
Qualitative evaluation
Rope
The visualization of rope trajectories in the real world and simulation with optimized parameters from the three different release points, upper, middle, and lower.
optimized sim real optimized sim real optimized sim real
Cloth
The visualization of cloth trajectories in the real world and the optimized simulation.
Cotton
Rubber
Abalation studies
Rope casting - The policy determines the momentary rope action so that the bottom end of the rope reaches the goal coordinates.
The policy trained by both Young's modulus and Poisson's ratio (Upper) outperforms the policy trained without parameters (lower).
The rope parameters and goal coordinates are the same in the column, but the results for actual reach coordinates are different.
Test1 Test2 Test3
Sim2Real
Rope releasing - The policy determines the release coordinates so that the bottom end of the rope reaches the goal coordinates.
Our method GenORM (upper), using only single demonstration, shows the almost same performance of πRD (lower) trained by 100 demonstrations. GenORM also outperforms other baselines. Please watch the demo video for more details.
Cloth spreading -The policy determines the optimal acceleration for spreading the cloth over the table.
The policy trained by both Young's modulus and Poisson's ratio (Upper) outperforms the policy trained without parameters (lower).
Towel Rubber
BibTex