Push_and_grasp

Self-supervised Learning for Joint Pushing and Grasping Policies in Highly Cluttered Environments

Yongliang Wang1*, Kamal Mokhtar1*, Cock Heemskerk2, Hamidreza Kasaei1

* These authors contributed equally to this work

1Department of Artificial Intelligence, Bernoulli Institute, University of Groningen

2Heemskerk Innovative Technology, Delft

IEEE International Conference on Robotics and Automation (ICRA 2024)

Paper

Code

Presentation

Abstract

Robotic systems often face challenges when attempting to grasp a target object due to interference from surrounding items. We propose a Deep Reinforcement Learning (DRL) method that develops joint policies for grasping and pushing, enabling effective manipulation of target objects within untrained, densely cluttered environments. In particular, a dual RL model is introduced, which presents high resilience in handling complicated scenes, reaching an average of 98% task completion in simulation and real-world scenes. To evaluate the proposed method, we conduct comprehensive simulation experiments in three distinct environments: densely packed building blocks, randomly positioned building blocks, and common household objects. Further, real-world tests are conducted using actual robots to confirm the robustness of our approach in various untrained and highly cluttered environments. The results from experiments underscore the superior efficacy of our method in both simulated and real-world scenarios, outperforming recent state-of-the-art methods.

Motivation

Method

Our system chooses between two actions: push and grasp, depending on environmental conditions. If grasp likelihood surpasses a threshold, it opts to grasp; otherwise, it pushes the target or adjacent objects approximately 1.3 cm to enhance subsequent grasp opportunities. Grasp/push coordinates, initially in the image domain, are later converted to the robot's frame for action.

System Overview: Within the Gazebo environment, an RGB-D camera and dual-arm UR5e robot are integrated. The camera transforms sensory data into orthographic projections, generating goal and scene masks. Using 360° rotation, the grasp and push nets assess heightmaps and the goal mask, outputting Q values to dictate push or grasp actions.

Experiments

To assess our push-grasping strategy, we conducted tests in both simulated and real-world scenarios. The objective was to compare its efficiency with established policies and verify its stability in real robotic systems, with minimal reduction in performance.

Simulation Experiments

Real Robot Experiments

BibTex

@inproceedings{wang2024self,

title={Self-supervised learning for joint pushing and grasping policies in highly cluttered environments},

author={Wang, Yongliang and Mokhtar, Kamal and Heemskerk, Cock and Kasaei, Hamidreza},

booktitle={2024 IEEE international conference on robotics and automation (ICRA)},

pages={13840--13847},

year={2024},

organization={IEEE}

}