HDR map reconstruction from a single LDR sky panoramic image for outdoor illumination estimation

Outdoor low dynamic range (LDR) panoramic images that contain the sun and sky are generally over-saturated because the sun is 10,000 times brighter than the regions that surround it. Because the luminance information in the region that contains the sun in these images is lost, it is difficult to identify the sun’s position and generate high dynamic range (HDR) environment maps that can be used to realistically relight virtual objects. Previous methods to reconstruct HDR maps did not consider that the sun covers a small area in an image but contains extremely high luminance values. These methods are therefore insufficient for estimating scene illumination. We propose a multi-faceted approach to reconstructing HDR maps from a single LDR sky panoramic image that considers the sun and sky regions separately. We encode an input image and transfer a multi-dimensional latent representation to two decoders, which reconstruct the luminance information in the sky and sun regions separately. To plausibly model sun illumination, we introduce two networks (Sunpose-net and Sunrad-net) that estimate the position and radiance of the sun. The generated sun radiance map is then merged with the output of the decoder that is responsible for sun regions. We demonstrated that the proposed method more plausibly reconstructs HDR maps than previous methods using the HDR-VDP-2.2 which measures the visual quality of reconstructed HDR maps against ground truth. The accuracy of the overall sun and sky illumination distribution in HDR maps reconstructed using the proposed method was evaluated using histogram distance measures.

G. Shin, K. YU, M. Mpabulungi, H. Hong, "HDR map reconstruction from a single LDR sky panoramic image for outdoor illumination estimation", IEEE ACCESS, 11, (2023). (SCIE)