Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds

Lirui Wang, Yu Xiang, Wei Yang, Arsalan Mousavian, and Dieter Fox

University of Washington, NVIDIA

The Conference on Robot Learning (CoRL), 2021

Paper, Code

6D robotic grasping beyond top-down bin-picking scenarios is a challenging task. Previous solutions based on 6D grasp synthesis with robot motion planning usually operate in an open-loop setting, which are sensitive to grasp synthesis errors. In this work, we propose a new method for learning closed-loop control policies for 6D grasping. Our policy takes a segmented point cloud of an object from an egocentric camera as input, and outputs continuous 6D control actions of the robot gripper for grasping the object. We combine imitation learning and reinforcement learning and introduce a goal-auxiliary actor-critic algorithm for policy learning. We demonstrate that our learned policy can be integrated into a tabletop 6D grasping system and a human-robot handover system to improve the grasping performance of unseen objects.


Ablation Studies in Simulation

We ablate our methods on simulation from several perspectives. We show expert demonstrations and object repositories as our data. We compare different policy models and with an open-loop system. We test on moving objects and show simple extensions of our method to cluttered-scene and placing. Finally, we show that our trained policy in simulation does have limitations where the same grasp can work in simulation but fail in real-world tests due to contact modeling.


Object Repository

Expert Demonstration


Model Comparison


Open-Loop Comparison


Closed-Loop Grasping


Real-World Contact Gap

Tabletop 6D Grasping in the Real World

We combine unseen object instance segmentation and GA-DDPG for 6D grasping of unseen objects.


We compare with a state-of-the-art open-loop method and show our policy can be used to improve the system for tabletop 6D grasping. We tested the methods on fixed (left) and varying (right) robot initial states on 9 YCB objects and 10 unseen objects.

Policy (GA-DDPG)


YCB objects with fix initial joints


YCB Objects with vary initial joints


Unseen Objects with fix initial joints


Unseen Objects with vary initial joints

Planning (6D GraspNet + OMG Planner)

YCB Objects with fixed initial joints

YCB Objects with vary initial joints


Unseen Objects with fixed initial joints

Unseen Objects with vary initial joints

Combined (Planning + Policy)


YCB Objects with fixed initial joints


YCB Objects with vary initial joints


Unseen Objects with fixed initial joints

Unseen Objects with vary initial joints

Human-to-robot Handover in the Real World

We combine human body tracking and object segmentation with GA-DDPG for human-to-robot handover.


We conduct experiments with different handover locations and object motion.


User Studies for Human-to-robot Handover

We test the handover system with users in the lab and most participants agree on the stability and reactivity of the system.


User 1


User 2


User 3


User 4


User 5


User 6