Detecting Object Affordances with Convolutional Neural Networks
Anh Nguyen, Dimitrios Kanoulas, Darwin G. Caldwell, and Nikos G. Tsagarakis
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016
Abstract:
We present a novel and real-time method to detect object affordances from RGB-D images. Our method trains a deep Convolutional Neural Network (CNN) to learn deep features from the input data in an end-to-end manner. The CNN has an encoder-decoder architecture in order to obtain smooth label predictions. The input data are represented as multiple modalities to let the network learn the features more effectively. Our method sets a new benchmark on detecting object affordances, improving the accuracy by 20% in comparison with the state-of-the-art methods that use hand-designed geometric features. Furthermore, we apply our detection method on a full-size humanoid robot (WALK-MAN) to demonstrate that the robot is able to perform grasps after efficiently detecting the object affordances.
Paper: PDF
Related Publications:
AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection
Thanh-Toan Do*, Anh Nguyen*, Ian Reid
ICRA 2018. (* equal contribution)
Anh Nguyen, Dimitrios Kanoulas, Darwin G. Caldwell, Nikos Tsagarakis
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017.
Video: