A Hierarchical Coding Scheme for Glasses-free 3D Displays Based On Scalable Hybrid Layered Representation of Real-World Light Fields
Joshitha R*, Mansi Sharma*
(* equal contribution)
IEEE International Conference on Systems, Man, and Cybernetics (IEEE SMC 2021), Melbourne, Australia
Abstract
This paper presents a novel hierarchical coding scheme for light fields based on transmittance patterns of low-rank multiplicative layers and Fourier disparity layers. The proposed scheme learns stacked multiplicative layers from subsets of light field views determined from different scanning orders. The multiplicative layers are optimized using a fast data-driven convolutional neural network (CNN). The essential factor for multiplicative layers representation, which has not been considered in previous compression approaches, is the origin of redundancy, i.e., the low-rank structure of light field data. The spatial correlation in layer patterns is exploited with varying low ranks through factorization derived from singular value decomposition on a Krylov subspace. Further, encoding with HEVC efficiently removes intra-view and inter-view correlation in low-rank approximated layers. The initial subset of
approximated decoded views from multiplicative representation is used to construct Fourier disparity layer (FDL) representation. The FDL model synthesizes the second subset of views identified by a pre-defined hierarchical prediction order. The correlations between the prediction residue of synthesized views are further eliminated by encoding the residual signal. The set of views obtained from decoding the residual is employed to refine the FDL model and predict the next view subsets with improved accuracy. This hierarchical procedure is repeated until all light field views are encoded. The critical advantage of the proposed hybrid layered representation and coding scheme is that it utilizes not just spatial and temporal redundancies, but efficiently exploits the strong intrinsic similarities among neighboring sub-aperture images in both horizontal and vertical directions as specified by different predication orders. Besides,
the scheme is flexible to realize a range of multiple bitrates at the decoder within a single integrated system. The compression performance analyzed with real light field shows substantial bitrate savings, maintaining good reconstruction quality.
SMC21_0415_FI.pdfCitation
J. Ravishankar and M. Sharma, "A Hierarchical Coding Scheme for Glasses-free 3D Displays Based on Scalable Hybrid Layered Representation of Real-World Light Fields," 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021, pp. 3491-3498, doi: 10.1109/SMC52423.2021.9658584.
