This page described the format of the COB-3D-v2 dataset (download)
The tar file contains the following directory structure:
v2/
|-- dset.json
|-- scenes/
|-- {scene_id}.npz
|-- meshes/
|-- {mesh_id}.stl
The dataset split is contained in v2/dset.json. This is a json file in the format:
{
"train": [
"scene_id_1",
"scene_id_2",
...,
],
"val: [
"scene_id_X",
...,
],
}
There are 6955 scenes in total (6259 train, 696 val).
Each scene is contained in an NPZ file: v2/scenes/{scene_id}.npz
The structure of each NPZ file is outlined below. The mesh_ids, obj_poses, voxel_grid fields were added since the v1 release to enable training shape completion models.
v2/scenes/{scene_id}.npz:
|-- rgb: The rendered RGB image.
Shape (3, H, W), dtype float32, values scaled to [0, 1].
|-- intrinsic: The camera intrinsics.
Shape (3, 3), dtype float32.
|-- depth_map: The rendered depth map corresponding to `rgb`.
Shape (H, W), dtype float32.
|-- normal_map: The rendered normal map corresponding to rgb`.
Shape (3, H, W), dtype float32.
|-- near_plane: The minimum depth value of the scene's working volume.
Scalar, float32.
|-- far_plane: The maximum depth value of the scene's working volume.
Scalar, float32.
|-- segm/
|-- boxes: 2D bounding boxes for each object in the scene.
Shape (N_objects, 4), dtype float32.
These are pixel coordinates relative to `rgb`.
The box format is `[x_low, y_low, x_high, y_high]`
|-- masks: Binary masks for each object in the scene.
Shape (N_objects, H, W), dtype bool.
|-- amodal_masks: Amodal instance masks for each object.
Shape (N_objects, H, W), dtype bool.
|-- bbox3d/
|-- poses: The pose of each object's 3D bounding box, as a 4x4 matrix.
This is the transform from the bbox frame to the camera frame.
Shape (N_objects, 4, 4), dtype float32.
|-- dimensions: The dimensions of each 3D bounding box.
Shape (N_objects, 3), dtype float32.
|-- corners: The corner points of each 3D bounding box, in the camera frame.
Shape (N_objects, 8, 3), dtype float32.
|-- mesh_ids: The mesh_id of each object.
These correspond to the STL files in `v2/meshes/{mesh_id}.stl`
List[str], length N_objects.
|-- obj_poses/
|-- poses: The pose of each mesh, as a 4x4 matrix.
This is the transform from the mesh frame to the camera frame.
Note that the mesh frame does not necessarily equal the bbox frame!
Shape (N_objects, 4, 4), dtype float32.
|-- scales: The scale of each mesh.
Shape (N_objects, 3), dtype float32.
|-- voxel_grid/
|-- voxels: The surface of each mesh, extracted into a voxel grid.
Shape (N_objects, n_voxels, n_voxel, n_voxels), dtype bool.
|-- extents: The extents of each object's voxel grid.
`voxel_grid/voxels[i]` span the cuboid `[-extents[i], extents[i]]`,
in the object frame `obj_poses/poses[i]`.
Shape (N_objects, 3), dtype float32.