A lot of tools have their own operating directions. Taking the knife as an example, in order to cut an apple, robots must grasp the knife in a way such that the blade faces to the apple. To properly manipulate a task i.e. cutting the paper, robots needs to not only recongize the tool's category and affordance, but they must estimate its pose or keypoints. Affordance determines the type of task, while keypoints determine how to manipulate the tool. Recognizing the affordances with keypoints of tools enables robots to perform a variety of manipulation tasks.

The RGB-D Affordance Keypoint Dataset contains RGB-D images and ground-truth annotations of object category, affordance segmentation and associated keypoints. It is based on the RGB-D Part Affordance Dataset (UMD Dataset) [1] and we made a series of modifications, which are reported detailedly in the README. It contains 104 common tools for robotic manipulation tasks, which come from original original UMD Dataset and self-collected dataset. The RGB-D Affordance Keypoint Dataset covers a wide range of daily tools which help deep neural networks generalize to noval objects. There are six affordances annotated on the surface of tools in total: grasp, cut, scoop, contain, pound and wrap-grasp (wgrasp) and five keypoints asscoiated for each affordance.

[1] Myers, Austin, Ching L. Teo, Cornelia Fermüller, and Yiannis Aloimonos. "Affordance detection of tool parts from geometric features." In 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 1374-1381. IEEE, 2015.

Downloads

• Dataset details: -README

• RGB-D Affordance Keypoint Dataset: - Dataset

• Dataset split file: - File