A Novel 3D-UNet Deep Learning Framework Based On High-Dimensional Bilateral Grid For Edge Consistent Single Image Depth Estimation
Mansi Sharma, Abheesht Sharma, Kadvekar Rohit Tushar, Avinash Panneer
In Proceedings of 2020 International Conference on 3D Immersion (IC3D 2020), Brussels, Belgium
Abstract
The task of predicting smooth and edge-consistent depth maps is notoriously difficult for single image depth estimation. This paper proposes a novel Bilateral Grid based 3D convolutional neural network, dubbed as 3DBG-UNet, that parameterize high dimensional feature space by encoding compact 3D bilateral grids with UNets and infers sharp geometric layout of the scene. Further, an another novel 3DBGES-UNet model is introduced that integrate 3DBG-UNet for inferring an accurate depth map given a single color view. The 3DBGES-UNet concatenate 3DBG-UNet geometry map with the inception network edge accentuation map and a spatial object’s boundary map obtained by leveraging semantic segmentation and train the UNet model with ResNet backbone. Both models are designed with a particular attention to explicitly account for edges or minute details. Preserving sharp discontinuities at depth edges is critical for many applications such as realistic integration of virtual objects in AR video or occlusion-aware view synthesis for 3D display applications. The proposed depth prediction network achieves state-of-the-art performance in both qualitative and quantitative evaluations on the challenging NYUv2-Depth data. The code and corresponding pre-trained weights will be made publicly available.
IC3D2020BG3DUNetDepthCRpaper.pdfCitation
M. Sharma, A. Sharma, K. R. Tushar and A. Panneer, "A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation," 2020 International Conference on 3D Immersion (IC3D), Brussels, Belgium, 2020, pp. 01-08, doi: 10.1109/IC3D51119.2020.9376327.
