Jianing Qian*, Thomas Weng*, Luxin Zhang, Brian Okorn, David Held

Robotics Institute, Carnegie Mellon University, Pittsburgh, PA

*These authors contributed equally.

[Paper] [Code]

This work has been accepted at IROS 2020.

A workshop version of this paper was previously accepted at RSS 2020 VLRRM.

Abstract

Cloth detection and manipulation is a common task in domestic and industrial settings, yet such tasks remain a challenge for robots due to cloth deformability. Furthermore, in many cloth-related tasks like laundry folding and bed making, it is crucial to manipulate specific regions like edges and corners, as opposed to folds. In this work, we focus on the problem of segmenting and grasping these key regions.

Our approach trains a network to segment the edges and corners of a cloth from a depth image, distinguishing such regions from wrinkles or folds. We also provide a novel algorithm for estimating the grasp location, direction, and directional uncertainty from the segmentation. We demonstrate our method on a real robot system and show that it outperforms baseline methods on grasping success.

IROS Presentation Video

Demonstration Video

Robustness Demonstration Video

RSS VLRRM Presentation Video

Acknowledgements

This work was supported by the National Science Foundation Smart and Autonomous Systems Program (IIS-1849154), the United States Air Force and DARPA under Contract No. FA8750-18-C-0092, LG Electronics, a NSF Graduate Research Fellowship (DGE-1745016), and a NASA Space Technology Research Fellowship (80NSSC17K0233). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.

Citation

If you found our work useful for your research, please use the following citation:

@InProceedings{Qian_2020_IROS,

author={Qian, Jianing and Weng, Thomas and Zhang, Luxin and Okorn, Brian and Held, David},

booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},

title={Cloth Region Segmentation for Robust Grasp Selection},

year={2020},

volume={},

number={},

pages={9553-9560},

doi={10.1109/IROS45743.2020.9341121}}

Changelog

9/17/2020: Added IROS 2020 presentation video.

7/31/2020: Added final IROS 2020 version of the paper with appendix. Updated demo videos. Updated author order.

7/24/2020: Added RSS 2020 VLRRM Presentation video.

7/11/2020: Updated to note IROS 2020 and RSS 2020 VLRRM Workshop acceptances. Added acknowledgements.

4/9/2020: Added code.

3/1/2020: Published original version of site.