LLF-Net

A Lightweight Depth Estimation Network for Wide-baseline Light Fields

Abstract

Existing traditional and ConvNet-based methods for light field depth estimation mainly work on narrow-baseline scenario. This paper explores the feasibility and capability of ConvNets to estimate depth in another promising scenario: wide-baseline light fields. Due to the deficiency of training samples, a large-scale and diverse synthetic wide-baseline dataset with labelled data is introduced for depth prediction tasks. Considering the practical goal for real-world applications, we design an end-to-end trained lightweight convolutional network to infer depths from light fields, called LLF-Net. The proposed LLF-Net is built by incorporating cost volume and attention modules, which allows variable angular light field inputs and enables to recover details at occlusion areas. Evaluations are made on the synthetic and real-world wide-baseline light fields, and experimental results show that the proposed network achieves the best performance when comparing to recent state-of-the-art methods. Further, we try to evaluate the LLF-Net on the popular narrow-baseline synthetic and real-world light field datasets, and the results show that the LLF-Net significantly improves the state-of-the-art performance of previous methods in narrow-baseline scenario.

A Wide-baseline Synthetic Light Field Dataset - WLF Dataset

Results

Coming soon

Reference

1. EPI-Shift: T. Leistner, H. Schilling, R. Mackowiak, S. Gumhold, and C. Rother, “Learning to think outside the box: Wide-baselinelight field depth estimation with epi-shift,” in 2019 International Conference on 3D Vision (3DV). IEEE, 2019, pp. 249–257.

2. LBDE-E: J. Shi, X. Jiang, and C. Guillemot, “A framework for learning depth from a flexible subset of dense and sparse light field views,”IEEE Transactions on Image Processing, vol. 28, no. 12,pp. 5867–5880, 2019.

Citation

@article{li2021lightweight,

title={A Lightweight Depth Estimation Network for Wide-baseline Light Fields.},

author={Li, Yan and Wang, Qiong and Zhang, Lu and Lafruit, Gauthier},

journal={IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society},

year={2021}

}