Merging-ISP: Multi-Exposure High Dynamic Range Image Signal Processing

Chaudhari P, Schirrmacher F, Maier A, Riess C, Köhler T (2021)

Publication Type: Conference contribution, Conference Contribution

Publication year: 2021

Event location: virtual

Abstract

High dynamic range (HDR) imaging combines multiple images with
different exposure times into a single high-quality image. The image signal processing
pipeline (ISP) is a core component in digital cameras to perform these
operations. It includes demosaicing of raw color filter array (CFA) data at different
exposure times, alignment of the exposures, conversion to HDR domain, and
exposure merging into an HDR image. Traditionally, such pipelines cascade algorithms
that address these individual subtasks. However, cascaded designs suffer
from error propagation, since simply combining multiple steps is not necessarily
optimal for the entire imaging task.
This paper proposes a multi-exposure HDR image signal processing pipeline
(Merging-ISP) to jointly solve all these subtasks. Our pipeline is modeled by a
deep neural network architecture. As such, it is end-to-end trainable, circumvents
the use of hand-crafted and potentially complex algorithms, and mitigates error
propagation. Merging-ISP enables direct reconstructions of HDR images of dynamic
scenes from multiple raw CFA images with different exposures. We compare
Merging-ISP against several state-of-the-art cascaded pipelines. The proposed
method provides HDR reconstructions of high perceptual quality and it
quantitatively outperforms competing ISPs by more than 1 dB in terms of PSNR.

Authors with CRIS profile

Franziska Schirrmacher Sonderforschungsbereich/Transregio 89 Invasives Rechnen Andreas Maier Lehrstuhl für Informatik 5 (Mustererkennung) Christian Rieß Lehrstuhl für Informatik 1 (IT-Sicherheitsinfrastrukturen)

Involved external institutions

e.solutions Gesellschaft mit beschränkter Haftung DE Germany (DE)

How to cite

APA:

Chaudhari, P., Schirrmacher, F., Maier, A., Riess, C., & Köhler, T. (2021). Merging-ISP: Multi-Exposure High Dynamic Range Image Signal Processing. In Proceedings of the DAGM German Conference on Pattern Recognition. virtual.

MLA:

Chaudhari, Prashant, et al. "Merging-ISP: Multi-Exposure High Dynamic Range Image Signal Processing." Proceedings of the DAGM German Conference on Pattern Recognition, virtual 2021.

BibTeX: Download