Bowen Wen, Wenzhao Lian, Kostas Bekris, Stefan Schaal
Task-relevant grasping is critical for industrial assembly, where downstream manipulation tasks constrain the set of valid grasps. Learning task-relevant grasping, however, is a challenging problem, since task-relevant grasp labels are hard to define and annotate. There are yet no widely accepted representations for modeling and off-the-shelf tools for performing task-relevant grasps. Existing solutions are also not scaling well to industrial settings, where object instances from a single category can significantly vary in terms of their dimensions. This work proposes a framework to learn task-relevant grasping for industrial objects without the need for time-consuming real-world data collection or manual annotation. To achieve this, the entire framework is trained solely in simulation, including supervised training with synthetic label generation and self-supervised, hand-object interaction. In addition, this paper proposes a novel, object-centric canonical representation at category level, which allows establishing dense correspondence across object instances and transferring task-relevant grasps to novel object instances. Extensive experiments on task-relevant grasping of densely cluttered industrial objects are conducted in both simulation and real world setups, demonstrating the effectiveness of the proposed framework. Code and data will be released upon acceptance.