BOSS: Bones, organs and skin shape model - Shetty, Karthik; Birkhold, Annette; Jaganathan, Srikrishna; Strobel, Norbert; Egger, Bernhard; Kowarschik, Markus; Maier, Andreas (Computers in Biology and Medicine, 2023)
A virtual anatomical model of a patient can be a valuable tool for enhancing clinical tasks such as workflow automation, patient-specific X-ray dose optimization, markerless tracking, positioning, and navigation assistance in image-guided interventions. For these tasks, it is highly desirable that the patient's surface and internal organs are of high quality for any pose and shape estimate. At present, the majority of statistical shape models (SSMs) are restricted to a small number of organs or bones or do not adequately represent the general population. To address this, we propose a deformable human shape and pose model that combines skin, internal organs, and bones, learned from CT images. By modeling the statistical variations in a pose-normalized space using probabilistic PCA while also preserving joint kinematics, our approach offers a holistic representation of the body that can be beneficial for automation in various medical applications. In an interventional setup, our model could, for example, facilitate automatic system/patient positioning, organ-specific iso-centering, automated collimation or collision prediction. We assessed our model's performance on a registered dataset, utilizing the unified shape space, and noted an average error of 3.6 mm for bones and 8.8 mm for organs. By utilizing solely skin surface data or patient metadata like height and weight, we find that the overall combined error for bone-organ measurement is 8.68 mm and 8.11 mm, respectively. To further verify our findings, we conducted additional tests on publicly available datasets with multi-part segmentations, which confirmed the effectiveness of our model. In the diverse TotalSegmentator dataset, the errors for bones and organs are observed to be 5.10mm and 8.72mm, respectively. Our work shows that anatomically parameterized statistical shape models can be created accurately and in a computationally efficient manner. The proposed approach enables the construction of shape models that can be directly integrated into to various medical applications.
CRIS-Team Scopus Importer:2023-09-08
Keywords: Forward kinematics; Human body; Mesh; Shape model

Deep Iterative 2D/3D Registration - Jaganathan, Srikrishna; Wang, Jian; Borsdorf, Anja; Shetty, Karthik; Maier, Andreas (Medical Image Computing and Computer Assisted Intervention – MICCAI 2021, 2021)
Deep Learning-based 2D/3D registration methods are highly robust but often lack the necessary registration accuracy for clinical application. A refinement step using the classical optimization-based 2D/3D registration method applied in combination with Deep Learning-based techniques can provide the required accuracy. However, it also increases the runtime. In this work, we propose a novel Deep Learning driven 2D/3D registration framework that can be used end-to-end for iterative registration tasks without relying on any further refinement step. We accomplish this by learning the update step of the 2D/3D registration framework using Point-to-Plane Correspondences. The update step is learned using iterative residual refinement-based optical flow estimation, in combination with the Point-to-Plane correspondence solver embedded as a known operator. Our proposed method achieves an average runtime of around 8s, a mean re-projection distance error of 0.60±0.40 mm with a success ratio of 97% and a capture range of 60 mm. The combination of high registration accuracy, high robustness, and fast runtime makes our solution ideal for clinical applications.

Deep Learning Compatible Differentiable X-ray Projections for Inverse Rendering - Shetty, Karthik; Birkhold, Annette; Strobel, Norbert; Egger, Bernhard; Jaganathan, Srikrishna; Kowarschik, Markus; Maier, Andreas (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

HOOREX: Higher Order Optimizers for 3D Recovery from X-Ray Images - Shetty, Karthik; Birkhold, Annette; Egger, Bernhard; Jaganathan, Srikrishna; Strobel, Norbert; Kowarschik, Markus; Maier, Andreas (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2024)
We propose a method to address the challenge of generating a 3D digital twin of a patient during an X-ray guided medical procedure from a single 2D X-ray projection image, a problem that is inherently ill-posed. To tackle this issue, we aim to infer the parameters of Bones, Organs and Skin Shape (BOSS) model, a deformable human shape and pose model. There are currently two main approaches for model-based estimation. Optimization-based methods try to iteratively fit a body model to 2D measurements, they produce accurate 2D alignments but are slow and sensitive to initialization. On the other hand, regression-based methods use neural networks to estimate the model parameters directly, resulting in faster predictions but often with misalignments. Our approach combines the benefits of both techniques by implementing a fully differentiable paradigm through the use of higher-order optimizers that only require the Jacobian, which can be determined implicitly. The network was trained on synthetic CT and real CBCT image data, ensuring view independence. We demonstrate the potential clinical applicability of our method by validating it on multiple datasets covering diverse anatomical regions, and achieving an error of 27.98 mm.
CRIS-Team Scopus Importer:2023-12-15

Learning the Update Operator for 2D/3D Image Registration - Jaganathan, Srikrishna; Wang, Jian; Borsdorf, Anja; Maier, Andreas (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

PLIKS: A Pseudo-Linear Inverse Kinematic Solver for 3D Human Body Estimation - Shetty, Karthik; Birkhold, Annette; Jaganathan, Srikrishna; Strobel, Norbert; Kowarschik, Markus; Maier, Andreas; Egger, Bernhard (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2023)
We introduce PLIKS (Pseudo-Linear Inverse Kinematic Solver) for reconstruction of a 3D mesh of the human body from a single 2D image. Current techniques directly regress the shape, pose, and translation of a parametric model from an input image through a non-linear mapping with minimal flexibility to any external influences. We approach the task as a model-in-the-loop optimization problem. PLIKS is built on a linearized formulation of the parametric SMPL model. Using PLIKS, we can analytically reconstruct the human model via 2D pixel-aligned vertices. This enables us with the flexibility to use accurate camera calibration information when available. PLIKS offers an easy way to introduce additional constraints such as shape and translation. We present quantitative evaluations which confirm that PLIKS achieves more accurate reconstruction with greater than 10% improvement compared to other state-of-the-art methods with respect to the standard 3D human pose and shape benchmarks while also obtaining a reconstruction error improvement of 12.9 mm on the newer AGORA dataset.
CRIS-Team Scopus Importer:2023-11-17
Keywords: 3D from single images

Self-Supervised 2D/3D Registration for X-Ray to CT Image Fusion - Jaganathan, Srikrishna; Kukla, Maximilian; Wang, Jian; Shetty, Karthik; Maier, Andreas (2023)
Deep Learning-based 2D/3D registration enables fast, robust, and accurate X-ray to CT image fusion when large annotated paired datasets are available for training. However, the need for paired CT volume and X-ray images with ground truth registration limits the applicability in interventional scenarios. An alternative is to use simulated X-ray projections from CT volumes, thus removing the need for paired annotated datasets. Deep Neural Networks trained exclusively on simulated X-ray projections can perform significantly worse on real X-ray images due to the domain gap. We propose a self-supervised 2D/3D registration framework combining simulated training with unsupervised feature and pixel space domain adaptation to overcome the domain gap and eliminate the need for paired annotated datasets. Our framework achieves a registration accuracy of 1.83 ± 1.16 mm with a high success ratio of 90.1% on real X-ray images showing a 23.9% increase in success ratio compared to reference annotation-free algorithms.<br
Keywords: Training; Representation learning; Deep learning; Computer vision; Computed tomography; Neural networks; Self-supervised learning

Training Deep Learning Models for 2D Spine X-rays Using Synthetic Images and Annotations Created from 3D CT Volumes - Sukesh, Richin; Fieselmann, Andreas; Jaganathan, Srikrishna; Shetty, Karthik; Kaergel, Rainer; Kordon, Florian; Kappler, Steffen; Maier, Andreas (Bildverarbeitung für die Medizin 2022, 2022)
When training deep learning models in the medical domain, one is always burdened with the task of obtaining reliable medical data annotated by experts. However, the availability of annotated data is often limited. To overcome such limitations, this paper addresses the idea of using synthetic spine X-ray data to train a deep learning model to aid in the detection of vertebrae. For this purpose, a pipeline for automatic generation of synthetic datasets comprising synthetic Xray images and their corresponding annotations is developed and evaluated. The results of these experiments show improvements in detection rates of the model when synthetic X-ray data is added to the training dataset.<br
Keywords: X-ray Imaging; Spine; Object Detection; Synthetic Data