RCTA Object Keypoints Dataset

The RCTA Object Keypoints Dataset (RCTA-OKD) is a dataset of six unique classes of object from the Robotics Collaborative Technology Alliance with their corresponding keypoints.

Dataset Download

Here

Code

We additionally provide source code for keypoint based pose optimization.

Training code Pose Estimation Visualization Scripts [Coming soon]

Dataset Format

From the dataset root directory, the dataset is split into 5 sub-components. Each sub-directory has a structure of the format: <class_id>/<sequence_id>.

Raw

The raw directory contains the collected color and depth frames, and associated camera intrinsic information.

Models

The models directory contains the reconstructed scene models (PLY format) and associated estimated sensor poses.

Open3D can be used to visualize a model:

from pathlib import Path

import numpy as np

import open3d as o3d

DATASET_DIR = '/data/rcta_object_keypoints_dataset'

model_path = Path(DATASET_DIR) / 'models'

model_paths = sorted([mp for mp in model_path.glob('**/*.ply')])

np.random.shuffle(model_paths) # Keep it lively

pcd = o3d.io.read_point_cloud(str(model_paths[0]))

o3d.visualization.draw_geometries([pcd])

Generated

The primary contents of the generated directory are the rgb and depth directories, which are the depth images generated by projecting the reconstructed scene models to each estimated camera pose. This directory also contains the output from the 3D annotator tool described in the paper.

Annotated

The annotated directory contains the annotation files that define the keypoint locations in each image.

Within each annotation directory are a single file named data.json where each name field corresponds to the filename of the collected image.

A subset of image frames from each class were annotated by hand for comparison with projected annotations. These manual annotations are given by the keypoints_<class_id>_manual directory.

To compare the automatically projected annotations with manual annotations, directories with match in the name are created, in which the annotated frames in those directories match the annotated frames from the associated `manual` directory.

Finally, the dense annotations are given by directories of the format keypoints_<class>_<mode>, where <mode> is used to describe the algorithm used to generate the kepoints. The mode values include naive automatic keypoint projection (auto), model-based refinement (icp), or keypoint-based refinement(refined). We quantitatively and qualitatively determined model-based refinement to be the best approach of the three.

Viz

Finally the viz directory shows a sample of the annotation files projected onto the respective images. There is only one sample sequence included here, the remaining visualizations can be generated using the scripts provided.