1. Overview

The ThuHand17 binocular hand pose dataset contains 117371 frames of binocular images captured from 8 subjects in training set, with 16 kinds of basic poses and lots of transitional poses between adjacent basic ones. One of the subjects performed several kinds of basic poses, while the other seven subjects performed all the 16 basic poses and the transitional ones. The test set contains 10000 frames of 2 subjects, different from all samples in training set. The images are captured by Leap Motion, under the resolution of 640x480 for both left and right images. 3D positions of the five fingertips and the wrist are annotated using 3D Guidance trakSTAR.

In "Data.zip", "S00" to "S07" are the samples of the 8 subjects in the training set, and "Test00" and "Test01" make up the test set. Files with names of "frame*_left.png" and "frame*_right.png" are left images and right images respectively, and the annotated results (three coordinates of the six magnetic sensors attached to the back of the six joints, in the magnetic coordinate system in centimeters) are saved in "frame*.txt", where the symbol "*" denotes the frame number.

We provide the list of prefix of filenames (without suffix "_left.png" or "_right_png") in "train_image.txt" and "test_image.txt".

Annotations from "frame*.txt" are transformed into pixel coordinates, saved in "train_label.txt" and "test_label.txt", in the same order as "train_image.txt" and "test_image.txt" separately. Each row contains the pixel coordinates of the six joints in binocular images, in the order of Wrist, Thumb.T, Index.T, Middle.T, Ring.T and Pinky.T. Every three numbers are the "u_l, u_r, v" coordinates of one joint (horizontal coordinates in left, right image, and vertical coordinate).

In "src" folder, there are three MATLAB scripts: "projectJoints.m" for projecting points onto image planes, "generate_lists.m" for generating image file lists and transformed labels of training set and test set, "visualize_labeled_images.m" for visualization of binocular images in the dataset with the six joints in color.

"calibration.txt" contains the imaging parameters of Leap Motion, which are used in "projectJoints.m".

For usage, please unzip "Data.zip" first.

2. Calibration

The annotated results in "frame*.txt" are 3D coordinates in the magnetic coordinate system, which is different from the coordinate system of Leap Motion. During our data sampling procedure, the electromagnetic transmitter of the TrakSTAR system and the Leap Motion camera were in the parallel position and never changed. So a sensor was used to measure the position of the origin point of the Leap Motion coordinate system, and then coordinates in the Leap Motion coordinate system can be obtained by translating the ones in the magnetic system.

Specifically, the transformation can be done following the equations below.

x = y' * 10 - 10.29

y = z' * 10 + 28.69

z =-x' * 10 + 272.58

where [x', y', z'] and [x, y, z] are the coordinates in the magnetic system (in "frame*.txt", in centimeters) and the Leap Motion system (in millimeters) separately.

3. Citing the dataset

@inproceedings{wei2017two,

title={Two-stream binocular network: Accurate near field finger detection based on binocular images},

author={Wei, Yi and Wang, Guijin and Zhang, Cairong and Guo, Hengkai and Chen, Xinghao and Yang, Huazhong},

booktitle={Visual Communications and Image Processing (VCIP), 2017 IEEE},

pages={1--4},

year={2017},

organization={IEEE}

}

4. Download

https://drive.google.com/drive/folders/1NbiX66PrT2PS9ryFLwjKNlFiHqxiwNyx?usp=sharing

5. Publication

Two-stream binocular network: Accurate near field finger detection based on binocular images

Yi Wei ; Guijin Wang ; Cairong Zhang ; Hengkai Guo ; Xinghao Chen ; Huazhong Yang.

Visual Communications and Image Processing (VCIP), 2017 IEEE

6. Contact

Team:

Yi Wei; Guijin Wang; Cairong Zhang; Hengkai Guo; Xinghao Chen; Huazhong Yang

Department of Electronic Engineering, Tsinghua University, Beijing 100084, China

For questions about the dataset, contact Cairong Zhang: zcr17@mails.tsinghua.edu.cn.