Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation
Xinghao Chen, Guijin Wang, Hengkai Guo, Cairong Zhang
Department of Electronic Engineering, Tsinghua University, Beijing, China
Neurocomputing, 2020
Introduction
Hand pose estimation from single depth images is an essential topic in computer vision and human computer interaction. Despite recent advancements in this area promoted by convolutional neural networks, accurate hand pose estimation is still a challenging problem. In this paper we propose a novel approach named as Pose guided structured Region Ensemble Network (Pose-REN) to boost the performance of hand pose estimation. Under the guidance of an initially estimated pose, the proposed method extracts regions from the feature maps of convolutional neural network and generates more optimal and representative features for hand pose estimation. The extracted feature regions are then integrated hierarchically according to the topology of hand joints by tree-structured fully connections to regress the refined hand pose. The final hand pose is obtained by an iterative cascaded method. Comprehensive experiments on public hand pose datasets demonstrate that our proposed method outperforms state-of-the-art algorithms.
Paper
Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation
Xinghao Chen, Guijin Wang, Hengkai Guo, Cairong Zhang
Neurocomputing, 2020
[arXiv] [ScienceDirect] [Code]
Results
Predicted labels: [NYU] [ICVL] [MSRA]
All labels are in the format of (u, v, d) where u and v are pixel coordinates. See [awesome-hand-pose-estimation] for more detailed comparisons.
NYU
ICVL
MSRA
Demo
We provide some live demos for different pre-trained models (ICVL, MSRA, NYU and Hands17) below. Depth images are captured from a Intel Realsense SR300 camera and the hand region is cropped from the original image using a naive depth thresholding segmentation. We then use Pose-REN to predict 3D coordinates of hand pose.
ICVL
MSRA
NYU
Hands17
Citation
@article{chen2020pose,
title={Pose guided structured region ensemble network for cascaded hand pose estimation},
author={Chen, Xinghao and Wang, Guijin and Guo, Hengkai and Zhang, Cairong},
journal={Neurocomputing},
volume={395},
pages={138--149},
year={2020},
publisher={Elsevier}
}