Real-to-Sim Parameter Learning for Deformable Packages Using High-Fidelity Simulators for Robotic Manipulation
Omey M. Manyar*, Hantao Ye*, Siddharth Mayya, Fan Wang and Satyandra K. Gupta
*Equal Contribution
Embedded Files
Abstract: Deformable packages are becoming increasingly prevalent in the logistics and warehouse industry, demanding robotic manipulation strategies that are robust and adaptive. Unlike rigid objects, these packages undergo significant shape changes under external forces, making their handling more complex. These deformable packages are often manipulated using suction cups and contain internal objects that shift during transport, introducing additional complexity. To ensure safe and efficient robotic manipulation at scale, high-fidelity simulation is crucial to support automated robot trajectory generation. In this work, we present a physics-driven simulation framework that accurately models multi-object deformable packages, capturing package deformations, suction-cup interactions, and internal object dynamics. Our approach employs a hyperelastic model to accurately simulate package-object interactions, including contact forces, suction-cup interactions, friction, and material deformation. By leveraging high-fidelity physics, we optimize simulation parameters using real-world trajectory and package deformation data, ensuring an accurate representation of package behavior. Running in real-time, our framework reduces real-to-sim discrepancies, making it viable for real-world deployment. Additionally, we develop a parallelized simulation environment for large-scale reinforcement learning and trajectory optimization, paving the way for scalable, efficient, and adaptable robotic manipulation of deformable packages in diverse warehouse settings.
Abstract: Deformable packages are becoming increasingly prevalent in the logistics and warehouse industry, demanding robotic manipulation strategies that are robust and adaptive. Unlike rigid objects, these packages undergo significant shape changes under external forces, making their handling more complex. These deformable packages are often manipulated using suction cups and contain internal objects that shift during transport, introducing additional complexity. To ensure safe and efficient robotic manipulation at scale, high-fidelity simulation is crucial to support automated robot trajectory generation. In this work, we present a physics-driven simulation framework that accurately models multi-object deformable packages, capturing package deformations, suction-cup interactions, and internal object dynamics. Our approach employs a hyperelastic model to accurately simulate package-object interactions, including contact forces, suction-cup interactions, friction, and material deformation. By leveraging high-fidelity physics, we optimize simulation parameters using real-world trajectory and package deformation data, ensuring an accurate representation of package behavior. Running in real-time, our framework reduces real-to-sim discrepancies, making it viable for real-world deployment. Additionally, we develop a parallelized simulation environment for large-scale reinforcement learning and trajectory optimization, paving the way for scalable, efficient, and adaptable robotic manipulation of deformable packages in diverse warehouse settings.
Sim vs Real Performance
Sim vs Real Performance
Simulation Model
Simulation Model
Multi-Object Model:
Multi-Object Model:
Hyper-elastic Model of Deformable Package
Suction Cup Constrained via Spring and Damper
Internal Rigid Object
Simulation with Varying Parameters
Simulation with Varying Parameters
Simulation with Varying Particle Sizes
Simulation with Varying Particle Sizes
N = 286 (Too Small, Object Falls Off)
N = 608
N = 1050
N = 1612
Parallelized Environment for Scalable Training
Parallelized Environment for Scalable Training
Parallel Execution in Nvidia Warp
Parallel Execution in Nvidia Warp
Ability to launch multiple environments at once
Each environment can be stepped independently
Suitable for training RL policies
Real World Data Collection with Motion Capture
Real World Data Collection with Motion Capture
Page updated
Report abuse