ICCV 2019
Kun Zhou1 Xiaoguang Han2 Nianjuan Jiang1 Kui Jia3 Jiangbo Lu1
1Shenzhen Cloudream Technology Co., Ltd. 2The Chinese University of Hong Kong (Shenzhen) 3South China University of Technology
Abstract
Estimating 3D human pose from a single image is a challenging task. This work attempts to address the uncertainty of lifting the detected 2D joints to the 3D space by introducing an intermediate state - Part-Centric Heatmap Triplets (HEMlets), which shortens the gap between the 2D observation and the 3D interpretation. The HEMlets utilize three joint-heatmaps to represent the relative depth information of the end-joints for each skeletal body part. In our approach, a Convolutional Network (ConvNet) is first trained to predict HEMlests from the input image, followed by a volumetric joint-heatmap regression. We leverage on the integral operation to extract the joint locations from the volumetric heatmaps, guaranteeing end-to-end learning. Despite the simplicity of the network design, the quantitative comparisons show a significant performance improvement over the best-of-grade method (by 20% on Human3.6M). The proposed method naturally supports training with "in-the-wild" images, where only weakly-annotated relative depth information of skeletal joints is available. This further improves the generalization ability of our model, as validated by qualitative comparisons on outdoor images.
Paper: [PDF];
Supplementary: [PDF]
Poster: [PDF]
DemoVideo1: [MP4]；DemoVideo2: [MP4]
New - Code: [GitHub link]
-----------------------------------------------------------------------------------------------------------------------------------------------------
New - Extended long version
"HEMlets PoSh: Learning Part-Centric Heatmap Triplets for 3D Human Pose and Shape Estimation, " IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 2021.
Paper: [PDF];
DemoVideo-1: [link]；(_3DPW_downtown_crossStreets_00.avi, without using any temporal smoothing)
DemoVideo-2: [link]；(_3DPW_downtown_sitOnStairs_00.avi, without using any temporal smoothing)
DemoVideo-3: [link]；(_3DPW_downtown_sitOnStairs_00_smplify.avi, without using any temporal smoothing, but using the SMPLify optimization as post-processing.)