A light-field display provides not only binocular depth sensation but also natural motion parallax with respect to head motion, which invokes a strong feeling of immersion. Such a display can be implemented with a set of stacked layers, each of which has pixels that can carry out light-ray operations (multiplication and addition). With this structure, the appearance of the display varies over the observed directions (i.e., a light field is produced) because the light rays pass through different combinations of pixels depending on both the originating points and outgoing directions. To display a specific 3-D scene, these layer patterns should be optimized to produce a light field that is as close as possible to that produced by the target three-dimensional scene. To deepen the understanding for this type of light field display, we focused on two important factors: light-ray operations carried out using layers and optimization methods for the layer patterns. Specifically, we compared multiplicative and additive layers, which are optimized using analytical methods derived from mathematical optimization or faster data-driven methods implemented as convolutional neural networks (CNNs). We compared combinations within these two factors in terms of the accuracy of light-field reproduction and computation time. Our results indicate that multiplicative layers achieve better accuracy than additive ones, and CNN-based methods perform faster than the analytical ones. We suggest that the best choice in terms of the balance between accuracy and computation speed is using multiplicative layers optimized using a CNN-based method.
Keita Maruyama, Keita Takahashi, Toshiaki Fujii: "Comparison of Layer Operations and Optimization Methods for Light Field Display", IEEE Access, DOI: 10.1109/ACCESS.2020.2975209, (2020.2.20). [IEEE Xplore]
Software written in Python [ Get our software (17.4 MB)].
This software includes implementations for layer pattern optimization (using an analytical method or a CNN-based method) for light field display (with additive or multiplicative layers). Please refer to "readme.txt" to find the terms of use and usage.
We propose a method of using a focal stack, i.e., a set of differently focused images, as the input for a novel light field display called a “tensor display.” Although this display consists of only a few light attenuating layers located in front of a backlight, it can be viewed from many directions (angles) simultaneously without the resolution of each viewing direction being sacrificed. Conventionally, a transmittance pattern is calculated for each layer from a light field, namely, a set of dense multi-view images (typically dozens) that are to be observed from different directions. However, preparing such a massive amount of images is often cumbersome for real objects. We developed a method that does not require a complete light field as the input; instead, a focal stack composed of only a few differently focused images is directly transformed into layer patterns. Our method greatly reduces the cost of acquiring data while also maintaining the quality of the output light field. We validated the method with experiments using synthetic light field datasets and a focal stack acquired by an ordinary camera.
Keita Takahashi, Yuto Kobayashi, Toshiaki Fujii: "From Focal Stack to Tensor Light-Field Display," IEEE Transactions on Image Processing, Vol. 27, No. 9, pp. 4571--4584, DOI: 10.1109/TIP.2018.2839263 (2018.9). [IEEE Xplore (Open Access)]
Our source codes written in C++. [ Get our software ] [ Get our software v2 ]
This software can generate semi-transparent layer patterns for a tensor display from a light field or a focal stack. A simulator of a tensor display is also included. To find the terms of use, please refer to "readme.txt". Version 2 is written using cv::Mat instead of the obsolete IplImage.
Supplementary video 2
Supplementary video 3
Keita Maruyama, Yasutaka Inagaki, Keita Takahashi, Toshiaki Fujii, Hajime Nagahara: "A 3-D Display Pipeline from Coded Aperture Camera to Tensor Light- Field Display", IEEE International Conference on Image Processing (ICIP), doi: 10.1109/ICIP.2019.8803741 2019. [IEEE Xplore]
Yuto Kobayashi, Shu Kondo, Keita Takahashi, Toshiaki Fujii: "A 3-D Display Pipeline: Capture, Factorize, and Display the Light Field of a Real 3-D Scene", ITE Transactions on Media Technology and Applications Vol. 5, No. 3, pp. 88--95, (2017.7.1) [ J-Stage (Open Access)]
Keita Takahashi, Yuto Kobayashi, Toshiaki Fujii: "Displaying Real World Light Fields using Stacked LCDs," The 23rd International Display Workshops in conjunction with Asia Display 2016, DES4/3D8 - 1, pp. 1300--1303, invited talk & Innovative Demonstration Session (I-DEMO), Fukuoka, Japan (2016.12.7-9).
Toyohiro Saito, Yuto Kobayashi, Keita Takahashi, Toshiaki Fujii: "Displaying Real-World Light-Fields with Stacked Multiplicative Layers: Requirement and Data Conversion for Input Multi-view Images," IEEE/OSA Journal of Display Technology, Volume 12, Issue 11, pp. 1290--1300, Print ISSN: 1551-319X, Online ISSN: 1558-9323, (2016.11) [ IEEE Xplorer ]