Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity

The following videos are the complete evaluation on the policy trained with ADR. Each video contains 1 test case with 10 episodes. The object-id and the success rate are included in the caption. We would like to emphasize that previous work has not shown such complexity of contact events and object generalization at the same time on a robot with a simple hand.

Two strategies: As discussed in Section 5.5 in the paper, there are generally two typical emergent strategies to finish this task: For both strategies, the robot first pushes the object against the wall and rotates it. In the "dropping" strategy, the robot lets the object drop on the right finger to catch it (see Box-0 episode 1,2,8,9,10 for examples). In the "standing" strategy, the robot rotates the object until it stands on its side and then reach the grasp (see Box-0 episode 4,5,6,7). One single policy network might demonstrate both strategies as shown in the videos below. Within the successful episodes of this policy, 32/78 are using the dropping strategy and 46/78 are using the standing strategy.

Surface friction: We want to highlight that the variation of surface friction places an important role in this task. For example, Box-2 has very different surface friction than the other boxes - we observe that Box-2 sometimes makes sticking contact with the wall at the beginning of the episode and then drop on the ground (see Box-2 episode 4 ,5,6,7,8). This leads to a different object pose distribution from Box-0 which mostly slides along the wall. As we will see in the next section, Box-2 is very challenging for the policy without ADR.

Non-box objects: The bottle and container test cases have very different shapes than boxes. The evaluations with these objects demonstrate out-of-distribution generalization on shapes with a policy that is trained only on boxes. We expect the policy works on non-box objects to the extent that the distribution of the object pose remains similar - Just as boxes, these objects are rotated until they stand on the side or leaning against the wall. We do not expect the policy to work with shapes such as a cylinder which will have a drastically different pose distribution when the robot interacts with it - we leave this to future work.

The following videos are the evaluations for a policy trained without ADR. This policy relies on the "standing strategy" much more often. Although the policy is trained on a fixed simulation environment, it still demonstrates generalization across object variations to some extent due to the nature of a closed-loop policy. Overall, the success rate is 45% lower than the policy trained with ADR shown above.

Here are the failure cases from the real robot evaluations above.

A failure case that happens before the initial contact:

• Missing initial contact: The robot is not able to reach the initial contact of the object to rotate it. This is mostly due to the noise in pose estimation and the variations in object dimension.

Failure cases that happen during the rotation:

• Object drops during rotation: The object drops to the table during rotation. One potential reason for this failure case is the insufficient delta rotation of the low-level controller due to the sim2real gap. In the "dropping" strategy, the policy is supposed to rotate object and then let it drop on the bottom finger. Before the dropping happens, the gripper needs to be rotated until the bottom finger is below the object. Otherwise, the bottom finger will not be able to catch the object. Another potential reason for this failure case is that the finger slips on the object.

• Repeated rotation: The robot repeatedly rotates and drops the object. This is different from the previous failure case because the robot moves down with the object at the same time. Our hypothesis for this failure case is that the policy gets stuck in a loop in the MDP.

• Joint limit: The robot hits a joint limit and the policy gets stuck at the joint limit.

Failure cases that happen after the rotation:

• Unexpected object dynamics: When the robot rotates the object, the object might move in unexpected ways. This mostly happens for the non-box objects.

• Stop reaching: Following the "standing" strategy, the policy successfully rotates the object to a stable pose on the side of the object. However, it cannot reach the final grasping pose. The gripper tries to move down to reach the pose but it collides with the object due to the unexpected object dimension.

• Timeout: Since we use a fixed episode length during evaluation, sometimes the policy does not have enough time to finish the task although it is very close to a success. This happens when the policy spends time to recover from some failed attempts at the beginning of the episode.

We include the counts of each failure cases and their video example below.

We use Iterative Closest Point (ICP) to estimate the pose of the object to be used as policy input. The following video shows the result of ICP across an episode of the real robot execution. We assume a given template point cloud of the object. For boxes, we construct the template by measuring the dimensions. For non-box objects, we use scanned models. We use ICP to align the template (red) to the observed partial point cloud of the scene (blue) from the RGB-D camera which includes both the gripper and the box. From the video, ICP is able to match the template to the observed point cloud reasonably well. One failure case of ICP is shown in video(d) below. When the object suddenly drops during execution as a result of policy failure, the object pose becomes too different from the previous timestep. This may create difficulties for ICP and may require a global registration algorithm.