Accurate Depth Map Estimation from a Lenslet Light field Camera

Abstract

This paper introduces an algorithm to estimate accurate depth map from a lenslet light field camera. Our algorithm estimates multi-view stereo correspondences at sub-pixel accuracy using a cost volume. Our key idea to build accurate costs is threefold.

First, sub-aperture images are displaced using the phase shift theorem. Second, gradient costs are adaptively aggregated using the angular coordinate of the light field. Third, feature correspondences between the sub-aperture images are utilized as an additional constraint. With the cost volume, a multi-label optimization propagates and corrects depth map at weak texture regions. Finally, we iteratively refine local depth map by fitting local quadratic function to estimate a non-discrete depth map. Since a micro-lens image contains unexpected distortions, we also present a method to correct the error. The effectiveness of our algorithm is demonstrated through challenging real world examples, with comparisons to the performance of state-of-the-art depth estimation algorithms.

What's new!!

18/01/18 Our new version algorithm based on learning-based matching costs has been accepted in IEEE TPAMI!!

17/07/26 We received a robustness champion award in depth estimation challenge at CVPR workshop on Light-Field for Computer Vision!!

14/07/07 We are going to present this work in ICVSS 2015 (Sicily, Italy)!!

14/05/15 We have uploaded dataset and source code (MATLAB 2014a Window version)!!

14/05/15 We are going to present this work in CVPR 2015 (Monday June 8, 3:30pm-6:00pm)

Depth from a Light Field Image with Learning-based Matching Costs

Hae-Gon Jeon, Jaesik Park, Gyeongmin Choe, Jinsun Park, Yunsu Bok, Yu-Wing Tai and In So Kweon

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Accepted

[PDF] [bibtex]

Accurate Depth Map Estimation from a Lenslet Light Field Camera

Hae-Gon Jeon, Jaesik Park, Gyeongmin Choe, Jinsun Park, Yunsu Bok, Yu-Wing Tai and In So Kweon

IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2015

[PDF] [Supplementary material] [Source code] [bibtex]

Related Link

Light-Field toolbox v.0.4 presented by Donald Dansereau.

Geometric light field camera calibration toolbox presented by Yunsu Bok.