An Automated Deep Learning Method for Tile AO/OTA Pelvic Fracture Severity Grading from Trauma whole-Body CT - Dreizin, David; Goldmann, Florian; LeBedis, Christina; Boscak, Alexis; Dattwyler, Matthew; Bodanapally, Uttam; Li, Guang; Anderson, Stephan; Maier, Andreas; Unberath, Mathias (Journal of Digital Imaging, 2021)
Admission trauma whole-body CT is routinely employed as a first-line diagnostic tool for characterizing pelvic fracture severity. Tile AO/OTA grade based on the presence or absence of rotational and translational instability corresponds with need for interventions including massive transfusion and angioembolization. An automated method could be highly beneficial for point of care triage in this critical time-sensitive setting. A dataset of 373 trauma whole-body CTs collected from two busy level 1 trauma centers with consensus Tile AO/OTA grading by three trauma radiologists was used to train and test a triplanar parallel concatenated network incorporating orthogonal full-thickness multiplanar reformat (MPR) views as input with a ResNeXt-50 backbone. Input pelvic images were first derived using an automated registration and cropping technique. Performance of the network for classification of rotational and translational instability was compared with that of (1) an analogous triplanar architecture incorporating an LSTM RNN network, (2) a previously described 3D autoencoder-based method, and (3) grading by a fourth independent blinded radiologist with trauma expertise. Confusion matrix results were derived, anchored to peak Matthews correlation coefficient (MCC). Associations with clinical outcomes were determined using Fisher’s exact test. The triplanar parallel concatenated method had the highest accuracies for discriminating translational and rotational instability (85% and 74%, respectively), with specificity, recall, and F1 score of 93.4%, 56.5%, and 0.63 for translational instability and 71.7%, 75.7%, and 0.77 for rotational instability. Accuracy of this method was equivalent to the single radiologist read for rotational instability (74.0% versus 76.7%, p = 0.40), but significantly higher for translational instability (85.0% versus 75.1, p = 0.0007). Mean inference time was < 0.1 s per test image. Translational instability determined with this method was associated with need for angioembolization and massive transfusion (p = 0.002–0.008). Saliency maps demonstrated that the network focused on the sacroiliac complex and pubic symphysis, in keeping with the AO/OTA grading paradigm. A multiview concatenated deep network leveraging 3D information from orthogonal thick-MPR images predicted rotationally and translationally unstable pelvic fractures with accuracy comparable to an independent reader with trauma radiology expertise. Model output demonstrated significant association with key clinical outcomes.
CRIS-Team Scopus Importer:2021-02-05
Keywords: Convolutional neural network; Deep learning; Pelvic fracture; Pelvic instability; Pelvic ring disruption; Tile classification

Automated CT pelvic fracture severity grading with deep learning: association with clinical outcomes - Dreizin, David; Goldmann, Florian; Chen, Tina; Unberath, Mathias (Conference on Machine Intelligence in Medical Imaging (C-MIMI), 2019)

Automated in-silico Phantoms for Atherosclerotic Plaque Classification Models from Spectral CT - Goldmann, Florian; Goldmann, Manuela; Wels, Michael; Allmendinger, Thomas; Sühling, Michael; Maier, Andreas (2024 IEEE Nuclear Science Symposium (NSS), Medical Imaging Conference (MIC) and Room Temperature Semiconductor Detector Conference (RTSD), 2024)
We present a novel approach to study atherosclerotic plaque classification models in spectral computed tomography. We propose an end-to-end pipeline for automatically generated, realistic synthetic phantoms of coronary and carotid plaques. Motivated by the need for a more precise coronary artery disease diagnosis, we aim to overcome current limitations such as inaccurate tissue quantification capabilities by conventional methods and insufficient real-world data availability for model validation. The phantoms from our pipeline are based on clinically validated parameters and material compositions to mimic real anatomical conditions. Our approach leverages a state-of-the-art, physics-based ray-tracing framework for simulated data acquisition, which is then processed using calibrated reconstruction software to ensure highly realistic simulated imaging. We conduct an experimental validation consisting of qualitative and quantitative analyses. Comparing signal-to-noise ratio and contrast-to-noise ratio shows similar values for both synthetic and real-world data. The clinical impact of this work is relevant, as it may enable the bootstrapping of deep learning models for plaque classification, potentially enhancing diagnostic accuracy and allow more differentiated treatment strategie

Enabling machine learning in X-ray-based procedures via realistic simulation of image formation - Unberath, Mathias; Zaech, Jan-Nico; Gao, Cong; Bier, Bastian; Goldmann, Florian; Lee, Sing Chun; Fotouhi, Javad; Taylor, Russell; Armand, Mehran; Navab, Nassir (International Journal of Computer Assisted Radiology and Surgery, 2019)
Purpose: Machine learning-based approaches now outperform competing methods in most disciplines relevant to diagnostic radiology. Image-guided procedures, however, have not yet benefited substantially from the advent of deep learning, in particular because images for procedural guidance are not archived and thus unavailable for learning, and even if they were available, annotations would be a severe challenge due to the vast amounts of data. In silico simulation of X-ray images from 3D CT is an interesting alternative to using true clinical radiographs since labeling is comparably easy and potentially readily available. Methods: We extend our framework for fast and realistic simulation of fluoroscopy from high-resolution CT, called DeepDRR, with tool modeling capabilities. The framework is publicly available, open source, and tightly integrated with the software platforms native to deep learning, i.e., Python, PyTorch, and PyCuda. DeepDRR relies on machine learning for material decomposition and scatter estimation in 3D and 2D, respectively, but uses analytic forward projection and noise injection to ensure acceptable computation times. On two X-ray image analysis tasks, namely (1) anatomical landmark detection and (2) segmentation and localization of robot end-effectors, we demonstrate that convolutional neural networks (ConvNets) trained on DeepDRRs generalize well to real data without re-training or domain adaptation. To this end, we use the exact same training protocol to train ConvNets on naïve and DeepDRRs and compare their performance on data of cadaveric specimens acquired using a clinical C-arm X-ray system. Results: Our findings are consistent across both considered tasks. All ConvNets performed similarly well when evaluated on the respective synthetic testing set. However, when applied to real radiographs of cadaveric anatomy, ConvNets trained on DeepDRRs significantly outperformed ConvNets trained on naïve DRRs (p< 0.01). Conclusion: Our findings for both tasks are positive and promising. Combined with complementary approaches, such as image style transfer, the proposed framework for fast and realistic simulation of fluoroscopy from CT contributes to promoting the implementation of machine learning in X-ray-guided procedures. This paradigm shift has the potential to revolutionize intra-operative image analysis to simplify surgical workflows.
CRIS-Team Scopus Importer:2023-03-08
Keywords: Artificial intelligence; Computer assisted surgery; Image guidance; Monte Carlo simulation; Robotic surgery; Segmentation

Learning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views - Bier, Bastian; Goldmann, Florian; Zaech, Jan Nico; Fotouhi, Javad; Hegeman, Rachel; Grupp, Robert; Armand, Mehran; Osgood, Greg; Navab, Nassir; Maier, Andreas; Unberath, Mathias (International Journal of Computer Assisted Radiology and Surgery, 2019)
Purpose: Minimally invasive alternatives are now available for many complex surgeries. These approaches are enabled by the increasing availability of intra-operative image guidance. Yet, fluoroscopic X-rays suffer from projective transformation and thus cannot provide direct views onto anatomy. Surgeons could highly benefit from additional information, such as the anatomical landmark locations in the projections, to support intra-operative decision making. However, detecting landmarks is challenging since the viewing direction changes substantially between views leading to varying appearance of the same landmark. Therefore, and to the best of our knowledge, view-independent anatomical landmark detection has not been investigated yet. Methods: In this work, we propose a novel approach to detect multiple anatomical landmarks in X-ray images from arbitrary viewing directions. To this end, a sequential prediction framework based on convolutional neural networks is employed to simultaneously regress all landmark locations. For training, synthetic X-rays are generated with a physically accurate forward model that allows direct application of the trained model to real X-ray images of the pelvis. View invariance is achieved via data augmentation by sampling viewing angles on a spherical segment of 120 ∘ × 90 ∘ . Results: On synthetic data, a mean prediction error of 5.6 ± 4.5 mm is achieved. Further, we demonstrate that the trained model can be directly applied to real X-rays and show that these detections define correspondences to a respective CT volume, which allows for analytic estimation of the 11 degree of freedom projective mapping. Conclusion: We present the first tool to detect anatomical landmarks in X-ray images independent of their viewing direction. Access to this information during surgery may benefit decision making and constitutes a first step toward global initialization of 2D/3D registration without the need of calibration. As such, the proposed concept has a strong prospect to facilitate and enhance applications and methods in the realm of image-guided surgery.
CRIS-Team Scopus Importer:2019-05-02
Keywords: 2D/3D registration; Anatomical landmarks; Convolutional neural networks; Landmark detection

Light field processing for media applications - Goldmann, Florian; Keinert, Joachim; Fink, Laura; Gul, Muhammad Shahzeb Khan; Jaschke, Tobias; Ziegler, Matthias; Baetz, Michel; Fößel, Siegfried; Prappacher, Nico (Immersive Video Technologies, 2023)

OrdinalNet: A Deep Learning-based Relative Image Quality Metric for Motion Compensation in CBCT - Goldmann, Manuela; Manhart, Michael; Preuhs, Alexander; Goldmann, Florian; Kowarschik, Markus; Maier, Andreas (IEEE Nuclear Science Symposium (NSS), Medical Imaging Conference (MIC) and Room Temperature Semiconductor Detector Conference (RTSD), 2024)
Interventional C-arm cone beam CT significantly reduces time-to-therapy for patients suffering from acute stroke. Extended acquisition times as compared to modern helical CT systems make rigid patient motion more likely, introducing a mismatch with the geometry alignment presupposed during reconstruction, leading to typical blurring or streaking artifacts. Different autofocus methods exist which aim to find a compensation trajectory and reestablish a motion-free reconstruction by optimizing a quality metric on the reconstructed images. However, depending on the type of implemented metric, these methods show diminished performance for larger motion amplitudes, often converging into local minima. To overcome these drawbacks, we present OrdinalNet, a deep learning model predicting a pseudo-binary score to assess motion artifact deterioration or improvement between two reconstructions resulting from slightly different motion trajectories. Our method eliminates the necessity for absolute artifact quantification in each image, along with establishing a relative ordering among data points which is sufficient for effective simplex-based optimization. The model is trained and evaluated on simulated motion applied to motion-free clinical acquisitions. When presented 1000 motion-affected pairs from previously unseen patients, it shows superior performance on a threshold-dependent binary classification task when compared to an entropy-based metric, resulting in areas under the curve of AUCONet =0.9295 and AUCEnt=0.63
Keywords: Cone-Beam Computed Tomography; Deep Learning; Motion estimation

Photon Counting-Based Metrics for Soft Tissue Differentiation in Coronary Plaque Analysis - Denzinger, S; Goldmann, Florian; Wels, Michael; Schöbinger, Max; Neumann, Dominik; Allmendinger, Thomas; Jürgens, Markus; Goldmann, Manuela; Sühling, Michael; Maier, Andreas (2024 IEEE Nuclear Science Symposium (NSS), Medical Imaging Conference (MIC) and Room Temperature Semiconductor Detector Conference (RTSD), 2024)
Photon counting CT is a promising novel improvement for coronary artery disease diagnostics, bypassing limitations of traditional CT angiography such as calcium blooming. We introduce two new methods integrated into our clinical software: The dual energy ratio and the effective Z map. Both aim to enhance tissue differentiation and plaque segmentation, especially for soft and adipose tissue. The dual energy ratio is easy to compute, but requires careful noise reduction and calibration. The effective Z map offers a more robust alternative, potentially reducing susceptibility to noise, but is still vulnerable to overlapping tissue types. While these methods could potentially revolutionize coronary artery disease assessment, successful clinical implementation and evaluation relies on high quality image denoising and available ground-truth data. Overcoming these challenges yields significant improvements in clinical outcomes by aiding in early detection and treatment of vulnerable plaque
Keywords: Measurement; Microwave integrated circuits; Semiconductor detectors; Computed tomography; Noise reduction; Noise; Software