Predicting Forward & Backward Facial Depth Maps from a Single RGB Image for Mobile 3D AR Application
Avinash P, Mansi Sharma
In Proceedings of 2019 International Conference on 3D Immersion (IC3D 2019), Brussels, Belgium, Part of Stereopsia
(World Immersion Forum)
PDF, CODE
Abstract
Cheap and fast 3D asset creation to enable AR/VR applications is a fast growing domain. This paper addresses a significant problem of reconstructing complete 3D information of a face in near real-time speed on a mobile phone. We propose a novel deep learning based solution to predict robust depth maps of a face, one forward facing and the other backward facing, from a single image from the wild. A critical contribution is that the proposed network is capable of learning the depths of the occluded part of the face too. This is achieved by training a fully convolutional neural network to learn the dual (forward and backward) depth maps, with a common encoder and two separate decoders. The 300 W-LP, a cloud point dataset, is used to compute the required dual depth maps from the training data.
IC3D2019DepthMobile3DARCRpaper.pdfPREDICTING FORWARD AND BACKWARD FACIAL DEPTH MAPS FROM A SINGLE RGB IMAGE.pdfCitation
P. Avinash and M. Sharma, "Predicting Forward & Backward Facial Depth Maps From a Single RGB Image For Mobile 3d AR Application," 2019 International Conference on 3D Immersion (IC3D), Brussels, Belgium, 2019, pp. 1-8. doi: 10.1109/IC3D48390.2019.8975899
BibTex
@INPROCEEDINGS{8975899,
author={P. {Avinash} and M. {Sharma}},
booktitle={2019 International Conference on 3D Immersion (IC3D)},
title={Predicting Forward Backward Facial Depth Maps From a Single RGB Image For Mobile 3d AR Application},
year={2019},
volume={},
number={},
pages={1-8},
keywords={Face 3D modeling;depth map estimation;CNN;3D reconstruction;real-time mobile augmented reality},
doi={10.1109/IC3D48390.2019.8975899},
ISSN={2379-1772},
month={Dec}}