Detailed Surface Geometry and Albedo Recovery from RGB-D Video

Under Natural Illumination

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

International Conference on Computer Vision (ICCV) 2017

Fig3 Comparison results on Backpack and Turtle model

Abstract:

In this paper we present a novel approach for depth map enhancement from an RGB-D video sequence. The basic idea is to exploit the photometric information in the color sequence. Instead of making any assumption about surface albedo or controlled object motion and lighting, we use the lighting variations introduced by casual object movement. We are effectively calculating photometric stereo from a moving object under natural illuminations. The key technical challenge is to establish correspondences over the entire image set. We therefore develop a lighting insensitive robust pixel matching technique that out-performs optical flow method in presence of lighting variations. In addition we present an expectation-maximization framework to recover the surface normal and albedo simultaneously, without any regularization term. We have validated our method on both synthetic and real datasets to show its superior performance on both surface details recovery and intrinsic decomposition.

Citation:

References:

[1] Q. Chen and V. Koltun. A simple model for intrinsic image decomposition with depth cues. In ICCV, 2013.

[2] J. Jeon, S. Cho, X. Tong, and S. Lee. Intrinsic image decomposition using structure-texture separation and surface normals. In ECCV, 2014.

[3] R. Or-El, G. Rosman, A. Wetzler, R. Kimmel, and A. M. Bruckstein. Rgbd-fusion: Real-time high precision depth recovery. In CVPR, 2015.