Object-Based Affordances Detection with Convolutional Neural Networks and Dense Conditional Random Fields
Anh Nguyen, Dimitrios Kanoulas, Darwin G. Caldwell, and Nikos G. Tsagarakis
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017
Abstract:
We present a new method to detect object affordances in real-world scenes using deep Convolutional Neural Networks (CNN), an object detector and dense Conditional Random Fields (CRF). Our system first trains an object detector to generate bounding box candidates from the images. A deep CNN is then used to learn the depth features from these bounding boxes. Finally, these feature maps are post-processed with dense CRF to improve the prediction along class boundaries. The experimental results on our new challenging dataset show that the proposed approach outperforms recent state-of-the-art methods by a substantial margin. Furthermore, from the detected affordances we introduce a grasping method that is robust to noisy data. We demonstrate the effectiveness of our framework on the full-size humanoid robot WALK-MAN using different objects in real-world scenarios.
Paper: PDF
Dataset: IIT-AFF Dataset
Related Publications:
AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection
Thanh-Toan Do*, Anh Nguyen*, Ian Reid
ICRA 2018. (* equal contribution)
Detecting Object Affordances with Convolutional Neural Networks
Anh Nguyen, Dimitrios Kanoulas, Darwin G. Caldwell, Nikos Tsagarakis
IROS 2016.
Video: