Yongliang Wang, Hamidreza Kasaei
Interactive Robot Learning Laboratory (IRL-Lab)
IEEE Robotics and Automation Letters, May 2025
Department of Artificial Intelligence, Bernoulli Institute, University of Groningen
Dense Clutter
The Target Object is Partially Occluded in Dense Clutter
The Target Object is Completely Occluded in Dense Clutter
Household Objects Dense Clutter
Robotic grasping in densely cluttered environments is challenging due to scarce collision-free grasp affordances. Non-prehensile actions can increase feasible grasps in cluttered environments, but most research focuses on single-arm rather than dual-arm manipulation. Policies from single-arm systems fail to fully leverage the advantages of dual-arm coordination. We propose a target-oriented hierarchical deep reinforcement learning (DRL) framework that learns dual-arm push-grasp synergy for grasping objects to enhance dexterous manipulation in dense clutter. Our framework maps visual observations to actions via a pre-trained deep learning backbone and a novel CNN-based DRL model, trained with Proximal Policy Optimization (PPO), to develop a dual-arm push-grasp strategy. The backbone enhances feature mapping in densely cluttered environments. A novel fuzzy-based reward function is introduced to accelerate efficient strategy learning. Our system is developed and trained in Isaac Gym and then tested in simulations and on a real robot. Experimental results show that our framework effectively maps visual data to dual push-grasp motions, enabling the dual-arm system to grasp target objects in complex environments. Compared to other methods, our approach generates 6-DoF grasp candidates and enables dual-arm push actions, mimicking human behavior. Results show that our method efficiently completes tasks in densely cluttered environments.
Single-arm
Dual-arm
We propose a self-supervised DRL framework for dual-arm robots to learn coordinated push-grasp actions in target-oriented tasks within densely cluttered environments. Our model processes RGB-D images to generate adaptive dual-arm push actions, enabling the grasping of target objects that are otherwise challenging to grasp directly. Our framework uses a large-scale DL model as the backbone for multi-degree-of-freedom (multi-DoF) grasping, integrating a custom CNN-based RL model, trained with PPO, to learn the push-grasping strategy. We introduce a novel fuzzy-based reward function to accelerate efficient strategy learning.
Dual-Arm Push-Grasping Learning Framework Overview: In the Isaac Gym environment, the target object is highlighted in green, and an RGB-D camera integrated with a dual-arm UR5e robot captures images, converting them into top-down height maps. RGB images are input into a grasp network pre-trained on GraspNet-1 Billion, which extracts features fed into a CNN-based RL model trained with PPO. This model produces a feature map, decoded by two motion decoders to generate actions within the environment. A fuzzy reward module provides feedback and guiding training in the light blue area.
Grasping Training
Push-Grasping Training
Push-Grasping Testing
We deployed the model to grasp target objects in clutter, including challenging cases with partial or full occlusions. We created diverse test scenarios with block structures and household objects to evaluate performance on familiar and novel objects.
Our Method
1 Push, 1 Grasp
1 Push, 1 Grasp
3 Push, 1 Grasp
1 Push, 1 Grasp
Other Methods
EPG: 11 Push, 1 Grasp
VPG: 16 Push, 2 Grasp
EPG: 18 Push, 1 Grasp
SPG: 8 Push, 1 Grasp
Our Method
2 Push, 1 Grasp
2 Push, 1 Grasp
2 Push, 1 Grasp
4 Push, 1 Grasp
Other Methods
EPG: 12 Push, 1 Grasp
EPG: 20 Push, 1 Grasp
SPG: 7 Push, 1 Grasp
SPG: 18 Push, 1 Grasp
Our Method
2 Push, 2 Grasp
3 Push, 1 Grasp
3 Push, Grasp
3 Push, 1 Grasp
Other Methods
EPG: 16 Push, 2 Grasp
SPG: 9 Push, 1 Grasp
SPG: 18 Push, 3 Grasp
EPG: 31 Push, 2 Grasp
16 designed scenes featuring household objects, with the goal object highlighted in green. Case 10: packed clutter with 8 objects; Case 11: dense clutter with 15 objects; Case 12: target partially occluded in dense clutter; Case 13: target fully occluded in dense clutter.
Examples of 6-DoF grasp poses generated by our model. The first two rows illustrate grasps using the single-arm robot, while the last two rows demonstrate grasps using the dual-arm robot.
The process of generating push paths: In the red box, the first column shows the color heightmap, the second displays the learned feature map, and the third presents the masked color heightmap. In the orange box, the first column contains the binary masked color heightmap, the second depicts the Gaussian-smoothed map, and the third highlights the peak-detected map. The yellow box represents the key points sorted by angle. In the green box, the first column shows the divided 2D paths (split based on pixel values, corresponding to positive and negative x directions in real-world coordinates, with pixel size halved). The second column visualizes the 3D paths, which integrate depth information, undergo a safety check considering workspace constraints, and are smoothed using the Savitzky-Golay method. The third column displays the final execution path.
8 examples demonstrating the calculation of target isolation rates for objects of various shapes. The first row of examples illustrates block-shaped objects, while the second row showcases household objects.
Unreachable Push Path Points
Grasp Multi-object
Imprecise Grasp Pose
