TL;DR: A re-interpretation of robotic grasping as neural surface rendering for learning global and local representations that enable effective any-view grasping

Summary: 

We introduce a novel, fully implicit 6DoF grasp detection method, NeuGraspNet, that re-interprets robotic grasping as surface rendering and predicts high-fidelity grasps from any random single viewpoint of a scene.  Our method exploits a learned implicit geometric scene representation to perform global and local surface rendering. This enables effective grasp candidate generation (using global features) and grasp quality prediction (using local features from a shared feature space). Our local neural surface rendering allows the model to encode the interaction between the robot's end-effector and the object's surface geometry. NeuGraspNet outperforms existing implicit and semi-implicit baseline methods in the literature. We demonstrate the real-world applicability of NeuGraspNet with a mobile manipulator robot, grasping in open spaces with clutter by rendering the scene, reasoning about graspable areas of different objects, and selecting grasps likely to succeed without colliding with the environment.