Fu-Jen Chu, Ruinian Xu and Patricio A. Vela

IVALab

Georgia Institute of Technology, GA, U.S.A.

Abstract: This paper presents a deep learning framework to predict the affordances of object parts for robotic manipulation. The framework segments affordance maps by jointly detecting and localizing candidate regions within an image. Rather than requiring annotated real-world images, the framework learns from synthetic data and adapts to real-world data without supervision. The method learns domain-invariant region proposal networks and task-level domain adaptation components with regularization on the predicted domains. A synthetic version of the UMD dataset is collected for auto-generating annotated, synthetic input data. Experimental results show that the proposed method outperforms an unsupervised baseline, and achieves performance close to state-of-the-art supervised approaches. An ablation study establishes the performance gap between the proposed method and the supervised equivalent (30%). Real-world manipulation experiments demonstrate use of the affordance segmentations for task execution, which achieves the same performance with supervised approaches.

RA-L with ICRA2019: paper link

Code: available on github

Dataset and toolkit: dataset, toolkit

[BibTex]