Pose-REN

Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation

Xinghao Chen, Guijin Wang, Hengkai Guo, Cairong Zhang

Department of Electronic Engineering, Tsinghua University, Beijing, China

Neurocomputing, 2020

xinghao.chen@outlook.com; wangguijin@tsinghua.edu.cn

Introduction

Hand pose estimation from single depth images is an essential topic in computer vision and human computer interaction. Despite recent advancements in this area promoted by convolutional neural networks, accurate hand pose estimation is still a challenging problem. In this paper we propose a novel approach named as Pose guided structured Region Ensemble Network (Pose-REN) to boost the performance of hand pose estimation. Under the guidance of an initially estimated pose, the proposed method extracts regions from the feature maps of convolutional neural network and generates more optimal and representative features for hand pose estimation. The extracted feature regions are then integrated hierarchically according to the topology of hand joints by tree-structured fully connections to regress the refined hand pose. The final hand pose is obtained by an iterative cascaded method. Comprehensive experiments on public hand pose datasets demonstrate that our proposed method outperforms state-of-the-art algorithms.

Paper

Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation

Xinghao Chen, Guijin Wang, Hengkai Guo, Cairong Zhang

Neurocomputing, 2020

[arXiv] [ScienceDirect] [Code]

Results

Predicted labels: [NYU] [ICVL] [MSRA]

All labels are in the format of (u, v, d) where u and v are pixel coordinates. See [awesome-hand-pose-estimation] for more detailed comparisons.

NYU

ICVL

MSRA

Demo

We provide some live demos for different pre-trained models (ICVL, MSRA, NYU and Hands17) below. Depth images are captured from a Intel Realsense SR300 camera and the hand region is cropped from the original image using a naive depth thresholding segmentation. We then use Pose-REN to predict 3D coordinates of hand pose.