Deep Chain HDRI: Reconstructing a High Dynamic Range Image from a Single Low Dynamic Range Image
Siyeong Lee
Gwon Hwan An
Suk-ju Kang
[Paper]
[Dataset]
Proposed deep chain HDRI architecture. Given an LDR image with a middle exposure value (EV 0), the EV ±1, ±2, ±3 images are inferred sequentially through the network. When the EV of the inferred image is far from the middle exposure value, the structure depth goes deeper than that of the image that has less exposure difference to infer the mapping relation more accurately. After finishing the process through the proposed network, a total of six LDR images are inferred to generate the LDR image stack. Using an HDRI synthesis technique (e.g. the method of Debevec and Malik, etc.), an HDR image is generated from the pseudo multi exposure stack.

Abstract

Recently, high dynamic range (HDR) imaging has attracted much attention as a technology to reflect human visual characteristics owing to the development of the display and camera technology. This paper proposes a novel deep neural network model that reconstructs an HDR image from a single low dynamic range (LDR) image. The proposed model is based on a convolutional neural network composed of dilated convolutional layers and infers LDR images with various exposures and illumination from a single LDR image of the same scene. Then, the final HDR image can be formed by merging these inference results. It is relatively simple for the proposed method to find the mapping between the LDR and an HDR with a different bit depth because of the chaining structure inferring the relationship between the LDR images with brighter (or darker) exposures from a given LDR image. The method not only extends the range but also has the advantage of restoring the light information of the actual physical world. The proposed method is an end-to-end reconstruction process, and it has the advantage of being able to easily combine a network to extend an additional range. In the experimental results, the proposed method shows quantitative and qualitative improvement in performance, compared with the conventional algorithms.

Dataset
  • VDS dataset: the dataset contains 96 scenes that cover a wide variety of content, e.g., natural scenes (both indoor and outdoor), wooded grounds, buildings, etc.
  • This paper describes the dataset and the methodology followed when collecting it in much greater detail. Please cite it if you intend to use this dataset.
  • All ground truth HDR images are synthesized by the HDR Toolbox implementation of Debevec and Malik 1997. And, all tone-mapped images are generated by the HDR Toolbox implementation of Reinhard et al. 2002. For all the ground truth camera response functions(.crf), we leveraged the approach of Debevec and Malik 1997 with the method of Grossberg and Nayar 2003 to sample pixels from the multi-exposure image stack (Please refer to the the lin_fun of the hdr toolbox, https://github.com/banterle/HDR_Toolbox/blob/master/source_code/Generation/DebevecCRF.m#L122).
  • Please refer to the link for evaluation-related code: CEVR
[Dataset]

Paper

S. Lee, G.H. An, S-J. Kang.
Deep Chain HDRI: Reconstructing a High Dynamic Range Image from a Single Low Dynamic Range Image
IEEE Access, 2018.
(hosted on IEEE Access)

[Bibtex]

Future works

Inverse tone mapping:
  • Siyeong Lee, Gwon Hwan An, and Suk-Ju Kang. "Deep recursive hdri: Inverse tone mapping using generative adversarial networks." proceedings of the European Conference on Computer Vision (ECCV). 2018. [Paper], [Code]
  • Siyeong Lee, So Yeon Jo, Gwon Hwan An, and Suk-Ju Kang. "Learning to Generate Multi-Exposure Stacks with Cycle Consistency for High Dynamic Range Imaging." IEEE Transactions on Multimedia (2020).
  • Jung Hee Kim*, Siyeong Lee*, and Suk-Ju Kang. End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images." Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 2021. [Paper], [Code]
  • So Yeon Jo, Siyeong Lee, and Namhyun Ahn, Suk-Ju Kang. "Deep Arbitrary HDRI: Inverse Tone Mapping with Controllable Exposure Changes." IEEE Transactions on Multimedia (2021).

Tone mapping:
  • Gwon Hwan An, Siyeong Lee, Yong-Deok Ahn, and Suk-Ju Kang. "Deep Tone‐mapped HDRNET for High Dynamic Range Image Restoration." SID Symposium Digest of Technical Papers. (2018).

Selected references to our work

Academic papers
  • Sy-Kai Chen, Hung-Lin Yen, Yu-Lun Liu, Min-Hung Chen, Hou-Ning Hu, Wen-Hsiao Peng, Yen-Yu Lin. "Learning Continuous Exposure Value Representations for Single-Image HDR Reconstruction." Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2023. [Paper], [Code]
  • Ning Zhang, Yuyao Ye, Yang Zhao, Ronggang Wang. "Revisiting the Stack-Based Inverse Tone Mapping." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
    • [Paper]

Acknowledgements

  • We are especially grateful to Junghee Kim for helping us reorganize the dataset for release.
  • We are especially grateful to Sy-Kai Chen for sharing the code that makes it easy to evaluate the VDS dataset.
  • This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.