Multi-Sensor Super-Resolution for Hybrid Range Imaging with Application to 3-D Endoscopy and Open Surgery

Köhler T, Haase S, Bauer S, Wasza J, Kilgus T, Maier-Hein L, Stock C, Hornegger J, Feussner H (2015)

Publication Type: Journal article

Publication year: 2015

Journal

Publisher: Elsevier

Book Volume: 24

Pages Range: 220-234

Journal Issue: 1

DOI: 10.1016/j.media.2015.06.011

Abstract

In this paper, we propose a multi-sensor super-resolution framework for hybrid imaging to super-resolve data from one modality by taking advantage of additional guidance images of a complementary modality. This concept is applied to hybrid 3-D range imaging in image-guided surgery, where high-quality photometric data is exploited to enhance range images of low spatial resolution. We formulate super-resolution based on the maximum a-posteriori (MAP) principle and reconstruct high-resolution range data from multiple low-resolution frames and complementary photometric information. Robust motion estimation as required for super-resolution is performed on photometric data to derive displacement fields of subpixel accuracy for the associated range images. For improved reconstruction of depth discontinuities, a novel adaptive regularizer exploiting correlations between both modalities is embedded to MAP estimation. We evaluated our method on synthetic data as well as ex-vivo images in open surgery and endoscopy. The proposed multi-sensor framework improves the peak signal-to-noise ratio by 2 dB and structural similarity by 0.03 on average compared to conventional single-sensor approaches. In ex-vivo experiments on porcine organs, our method achieves substantial improvements in terms of depth discontinuity reconstruction.

Authors with CRIS profile

Involved external institutions

Ruprecht-Karls-Universität Heidelberg Germany (DE) Technische Universität München (TUM) Germany (DE) Deutsches Krebsforschungszentrum (DKFZ) Germany (DE)

How to cite

APA:

Köhler, T., Haase, S., Bauer, S., Wasza, J., Kilgus, T., Maier-Hein, L.,... Feussner, H. (2015). Multi-Sensor Super-Resolution for Hybrid Range Imaging with Application to 3-D Endoscopy and Open Surgery. Medical Image Analysis, 24(1), 220-234. https://doi.org/10.1016/j.media.2015.06.011

MLA:

Köhler, Thomas, et al. "Multi-Sensor Super-Resolution for Hybrid Range Imaging with Application to 3-D Endoscopy and Open Surgery." Medical Image Analysis 24.1 (2015): 220-234.

BibTeX: Download