HOOREX: Higher Order Optimizers for 3D Recovery from X-Ray Images

Shetty K, Birkhold A, Egger B, Jaganathan S, Strobel N, Kowarschik M, Maier A (2024)

Publication Type: Conference contribution

Publication year: 2024

Journal

Lecture Notes in Computer Science Springer Verlag

Publisher: Springer Science and Business Media Deutschland GmbH

Book Volume: 14315 LNCS

Pages Range: 115-124

Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Event location: Honolulu, HI, USA

ISBN: 9783031476785

DOI: 10.1007/978-3-031-47679-2_9

Abstract

We propose a method to address the challenge of generating a 3D digital twin of a patient during an X-ray guided medical procedure from a single 2D X-ray projection image, a problem that is inherently ill-posed. To tackle this issue, we aim to infer the parameters of Bones, Organs and Skin Shape (BOSS) model, a deformable human shape and pose model. There are currently two main approaches for model-based estimation. Optimization-based methods try to iteratively fit a body model to 2D measurements, they produce accurate 2D alignments but are slow and sensitive to initialization. On the other hand, regression-based methods use neural networks to estimate the model parameters directly, resulting in faster predictions but often with misalignments. Our approach combines the benefits of both techniques by implementing a fully differentiable paradigm through the use of higher-order optimizers that only require the Jacobian, which can be determined implicitly. The network was trained on synthetic CT and real CBCT image data, ensuring view independence. We demonstrate the potential clinical applicability of our method by validating it on multiple datasets covering diverse anatomical regions, and achieving an error of 27.98 mm.

Authors with CRIS profile

Karthik Shetty Lehrstuhl für Informatik 5 (Mustererkennung) Bernhard Egger Professur für Cognitive Computer Vision (Stiftungsprofessur) Srikrishna Jaganathan Lehrstuhl für Informatik 5 (Mustererkennung) Andreas Maier Lehrstuhl für Informatik 5 (Mustererkennung)

Involved external institutions

Siemens AG DE Germany (DE)

How to cite

APA:

Shetty, K., Birkhold, A., Egger, B., Jaganathan, S., Strobel, N., Kowarschik, M., & Maier, A. (2024). HOOREX: Higher Order Optimizers for 3D Recovery from X-Ray Images. In Andreas K. Maier, Julia A. Schnabel, Pallavi Tiwari, Oliver Stegle (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 115-124). Honolulu, HI, USA: Springer Science and Business Media Deutschland GmbH.

MLA:

Shetty, Karthik, et al. "HOOREX: Higher Order Optimizers for 3D Recovery from X-Ray Images." Proceedings of the 1st International Workshop on Machine Learning for Multimodal Healthcare Data, ML4MHD 2023, Honolulu, HI, USA Ed. Andreas K. Maier, Julia A. Schnabel, Pallavi Tiwari, Oliver Stegle, Springer Science and Business Media Deutschland GmbH, 2024. 115-124.

BibTeX: Download