Where shall I touch? Vision-Guided Tactile Poking for Transparent Object Grasping

Paper

Dataset

Code for Grasping

Code for Rendering

Abstract

Transparent object grasping is still a challenging task for robots. The unique visual properties such as reflection, refraction make the existing depth cameras fail to produce accurate depth estimates. However, humans can handle the transparent object well by first observing its coarse profile and then poking an area of interest to get a fine profile. Inspired by this, in this paper we propose a novel framework of vision-guided tactile poking for transparent object grasping. In the proposed framework, a segmentation network is first used to predict the potential regions for a poke that will obtain good contact with the object but lead to minimal disturbances to the state of the object. A poke is then performed with a high-resolution GelSight tactile sensor, and the obtained tactile reading can provide an accurate profile of the contact area. With the obtained accurate object profiles from the pokes, a heuristic grasp can be planned at the end for grasping the transparent object. Extensive experiments demonstrate that our proposed segmentation network can accurately predict the potential poking region, and the vision-guided tactile poking can enhance the grasping success rate significantly.