Abstract

This paper presents a novel simulation platform, ZeMa, designed for robotic manipulation tasks concerning soft objects. Such simulation ideally requires three properties: two-way soft-rigid coupling, intersection-free guarantees, and frictional contact modeling, with acceptable runtime suitable for deep and reinforcement learning tasks. Current simulators often satisfy only a subset of these needs, primarily focusing on distinct rigid-rigid or soft-soft interactions. The proposed ZeMa prioritizes physical accuracy and integrates the incremental potential contact method, offering unified dynamics simulation for both soft and rigid objects. It efficiently manages soft-rigid contact, operating 75x faster than baseline tools with similar methodologies like IPC-GraspSim. To demonstrate its applicability, we employ it for parallel grasp generation, penetrated grasp repair, and reinforcement learning for grasping, successfully transferring the trained RL policy to real-world scenarios.

Video

Overview

We illustrate the pipeline of the grasp interface of ZeMa and a demo of grasping a piece of cloth and throwing it onto a rigid cone. 

Additional Results

Grasp Penetration Repairing

The reason why the dataset after repair still has a low success rate is that most grasp poses provided by DexGraspNet are not valid grasps. The robot hand and the object are posed to make contact with each other but do not guarantee a grasp pose. We demonstrate some failure cases and successful ones here. 

In the above figure, (a) and (b) are two valid grasps samples, while (c) and (d) are invalid ones. Grasps in (a) and (c) are the original results from DexGraspNet, where the severe mesh intersections are highlighted by yellow circles. (b) and (d) are intersection-free results repaired by ZeMa. Since the objects in (c) and (d) are too large and their curvatures are too small, it is difficult to generate valid grasps for them. 

Reinforcement Learning

Ablation Study

Softbody Runtime

Shearing Coefficient

IPC-Graspsim Issue

Using codes from the IPC-Graspsim Github repository to run the vase grasping scene, we found that it generates some instable results which are shown below. 

Implementation Details

PD Controll

Experiment Setup

To evaluate the sim-to-real gap of ZeMa, we executed two experiments using a gel elastomer with markers, both in reality and within ZeMa. In the first, a robot gripper, with an indenter affixed, grasps a cube and depresses the gel on a table by 0.5mm before rotating 0.3 rad vertically (illustrated in (a) below). We monitor the movement of markers on the elastomer using a camera positioned below and simulate the scenario in ZeMa for comparison. The subsequent experiment mirrors the first, but the gripper shifts 1 mm horizontally post-press instead of rotating (illustrated in (b) below). A calibrated camera captures real-world marker displacements, and an analogous camera in ZeMa captures simulated displacements. Tracking these markers, ZeMa's predicted errors were 0.78px and 1.94px, with relative errors of 5.1% and 5.5%, underscoring ZeMa's alignment with real-world outcomes. The experenmental setup is shown below.