Abstract: Is Medical Chest X-ray Data Anonymous? - Packhäuser, Kai; Gündel, Sebastian; Münster, Nicolas; Syben, Christopher; Christlein, Vincent; Maier, Andreas (Informatik aktuell, 2023)

Abstract: Simultaneous Estimation of X-ray Back-scatter and Forward-scatter using Multi-task Learning - Roser, Philipp; Zhong, Xia; Birkhold, Annette; Preuhs, Alexander; Syben-Leisner, Christopher; Hoppe, Elisabeth; Strobel, Norbert; Kowarschik, Markus; Fahrig, Rebecca; Maier, Andreas (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

Age Estimation on Panoramic Dental X-ray Images using Deep Learning - Wallraff, Sarah; Vesal, Sulaiman; Syben-Leisner, Christopher; Lutz, Rainer; Maier, Andreas (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-09

A Gentle Introduction to Deep Learning in Medical Image Processing - Maier, Andreas; Syben-Leisner, Christopher; Lasser, Tobias; Riess, Christian (2018)

A gentle introduction to deep learning in medical image processing [Eine sanfte Einführung in Tiefes Lernen in der Medizinischen Bildverarbeitung] - Maier, Andreas; Syben-Leisner, Christopher; Lasser, Tobias; Riess, Christian (Zeitschrift für Medizinische Physik, 2019)
This paper tries to give a gentle introduction to deep learning in medical image processing, proceeding from theoretical foundations to applications. We first discuss general reasons for the popularity of deep learning, including several major breakthroughs in computer science. Next, we start reviewing the fundamental basics of the perceptron and neural networks, along with some fundamental theory that is often omitted. Doing so allows us to understand the reasons for the rise of deep learning in many application domains. Obviously medical image processing is one of these areas which has been largely affected by this rapid progress, in particular in image detection and recognition, image segmentation, image registration, and computer-aided diagnosis. There are also recent trends in physical simulation, modeling, and reconstruction that have led to astonishing results. Yet, some of these approaches neglect prior knowledge and hence bear the risk of producing implausible results. These apparent weaknesses highlight current limitations of deep learning. However, we also briefly discuss promising approaches that might be able to resolve these problems in the future.
CRIS-Team Scopus Importer:2019-05-03
Keywords: Computer-aided diagnosis; Deep learning; Image reconstruction; Image registration; Image segmentation; Introduction; Machine learning; Physical simulation

Automatic thrombus detection in non-enhanced computed tomography images in patients with acute ischemic stroke - Löber, Patrick; Stimpel, Bernhard; Syben-Leisner, Christopher; Maier, Andreas; Ditt, Hendrik; Schramm, Peter; Raczkowski, Boy; Kemmling, Andre (Eurographics Proceedings 2017, 2017)
UnivIS-Import:2017-12-18:Pub.2017.tech.IMMD.IMMD5.automa_9

Cephalogram Synthesis and Landmark Detection in Dental Cone-Beam CT Systems - Huang, Yixing; Fan, Fuxin; Syben-Leisner, Christopher; Roser, Philipp; Mill, Leonid; Maier, Andreas (Medical Image Analysis, 2021)
Due to the lack of a standardized 3D cephalometric analysis methodology, 2D cephalograms synthesized from 3D cone-beam computed tomography (CBCT) volumes are widely used for cephalometric analysis in dental CBCT systems. However, compared with conventional X-ray film based cephalograms, such synthetic cephalograms lack image contrast and resolution, which impairs cephalometric landmark identification. In addition, the increased radiation dose applied to acquire the scan for 3D reconstruction causes potential health risks. In this work, we propose a sigmoid-based intensity transform that uses the nonlinear optical property of X-ray films to increase image contrast of synthetic cephalograms from 3D volumes. To improve image resolution, super resolution deep learning techniques are investigated. For low dose purpose, the pixel-to-pixel generative adversarial network (pix2pixGAN) is proposed for 2D cephalogram synthesis directly from two cone-beam projections. For landmark detection in the synthetic cephalograms, an efficient automatic landmark detection method using the combination of LeNet-5 and ResNet50 is proposed. Our experiments demonstrate the efficacy of pix2pixGAN in 2D cephalogram synthesis, achieving an average peak signal-to-noise ratio (PSNR) value of 33.8 with reference to the cephalograms synthesized from 3D CBCT volumes. Pix2pixGAN also achieves the best performance in super resolution, achieving an average PSNR value of 32.5 without the introduction of checkerboard or jagging artifacts. Our proposed automatic landmark detection method achieves 86.7% successful detection rate in the 2 mm clinical acceptable range on the ISBI Test1 data, which is comparable to the state-of-the-art methods. The method trained on conventional cephalograms can be directly applied to landmark detection in the synthetic cephalograms, achieving 93.0% and 80.7% successful detection rate in 4 mm precision range for synthetic cephalograms from 3D volumes and 2D projections, respectively.<br
Keywords: Deep learning; cephalogram synthesis; super resolution; landmark detection; perspective deformation learning

Deep autofocus with cone-beam CT consistency constraint - Preuhs, Alexander; Manhart, Michael; Roser, Philipp; Stimpel, Bernhard; Syben-Leisner, Christopher; Psychogios, Marios; Kowarschik, Markus; Maier, Andreas (Bildverarbeitung für die Medizin, 2020)

Deep learning-based patient re-identification is able to exploit the biometric nature of medical chest X-ray data - Packhäuser, Kai; Gündel, Sebastian; Münster, Nicolas; Syben, Christopher; Christlein, Vincent; Maier, Andreas (Scientific Reports, 2022)
With the rise and ever-increasing potential of deep learning techniques in recent years, publicly available medical datasets became a key factor to enable reproducible development of diagnostic algorithms in the medical domain. Medical data contains sensitive patient-related information and is therefore usually anonymized by removing patient identifiers, e.g., patient names before publication. To the best of our knowledge, we are the first to show that a well-trained deep learning system is able to recover the patient identity from chest X-ray data. We demonstrate this using the publicly available large-scale ChestX-ray14 dataset, a collection of 112,120 frontal-view chest X-ray images from 30,805 unique patients. Our verification system is able to identify whether two frontal chest X-ray images are from the same person with an AUC of 0.9940 and a classification accuracy of 95.55%. We further highlight that the proposed system is able to reveal the same person even ten and more years after the initial scan. When pursuing a retrieval approach, we observe an mAP@R of 0.9748 and a precision@1 of 0.9963. Furthermore, we achieve an AUC of up to 0.9870 and a precision@1 of up to 0.9444 when evaluating our trained networks on external datasets such as CheXpert and the COVID-19 Image Data Collection. Based on this high identification rate, a potential attacker may leak patient-related information and additionally cross-reference images to obtain more information. Thus, there is a great risk of sensitive content falling into unauthorized hands or being disseminated against the will of the concerned patients. Especially during the COVID-19 pandemic, numerous chest X-ray datasets have been published to advance research. Therefore, such data may be vulnerable to potential attacks by deep learning-based re-identification algorithms.<br

Deep Scatter Splines: Learning-Based Medical X-ray Scatter Estimation Using B-splines - Roser, Philipp; Birkhold, Annette; Preuhs, Alexander; Syben-Leisner, Christopher; Strobel, Norbert; Kowarschik, Markus; Fahrig, Rebecca; Maier, Andreas (The 6th International Conference on Image Formation in X-Ray Computed Tomography (CT-Meeting), 2020)

Deriving Neural Network Architectures Using Precision Learning: Parallel-to-Fan Beam Conversion - Syben-Leisner, Christopher; Stimpel, Bernhard; Lommen, Jonathan; Würfl, Tobias; Dörfler, Arnd; Maier, Andreas (40th German Conference on Pattern Recognition, GCPR 2018, 2019)
Keywords: Hybrid MRI/X-ray imaging; Precision learning; Machine learning

Detecting Anatomical Landmarks for Motion Estimation in Weight-Bearing Imaging of Knees - Bier, Bastian; Aschoff, Katharina; Syben-Leisner, Christopher; Unberath, Mathias; Levenston, Marc; Gold, Garry; Fahrig, Rebecca; Maier, Andreas (MACHINE LEARNING FOR MEDICAL IMAGE RECONSTRUCTION, MLMIR 2018, 2018)
CRIS-Team WoS Importer:2019-08-09

Fan-beam Projection Image Acquisition using MRI - Syben-Leisner, Christopher; Stimpel, Bernhard; Leghissa, Martino; Dörfler, Arnd; Maier, Andreas (3rd Conference on Image-Guided Interventions & Fokus Neuroradiologie, 2017)
UnivIS-Import:2017-12-18:Pub.2017.tech.IMMD.IMMD5.fanbea_5

Fan-to-Parallel Beam Conversion: Deriving Neural Network Architectures Using Precision Learning - Syben-Leisner, Christopher; Stimpel, Bernhard; Lommen, Jonathan; Würfl, Tobias; Dörfler, Arnd; Maier, Andreas (Informatik aktuell, 2019)
CRIS-Team Scopus Importer:2019-05-14

Fully-automatic CT data preparation for interventional X-ray skin dose simulation - Roser, Philipp; Birkhold, Annette; Preuhs, Alexander; Stimpel, Bernhard; Syben, Christopher; Strobel, Norbert; Kowarschik, Markus; Fahrig, Rebecca; Maier, Andreas (Informatik Aktuell Bildverarbeitung für die Medizin 2020 Algorithmen - Systeme - Anwendungen, 2020)
Recently, deep learning (DL) found its way to interventional X-ray skin dose estimation. While its performance was found to be acceptable, even more accurate results could be achieved if more data sets were available for training. One possibility is to turn to computed tomography (CT) data sets. Typically, computed tomography (CT) scans can be mapped to tissue labels and mass densities to obtain training data. However, care has to be taken to make sure that the different clinical settings are properly accounted for. First, the interventional environment is characterized by wide variety of table setups that are significantly different from the typical patient tables used in conventional CT. This cannot be ignored, since tables play a crucial role in sound skin dose estimation in an interventional setup, e.,g., when the X-ray source is directly underneath a patient (posterior-anterior view). Second, due to interpolation errors, most CT scans do not facilitate a clean segmentation of the skin border. As a solution to these problems, we applied connected component labeling (CCL) and Canny edge detection to (a) robustly separate the patient from the table and (b) to identify the outermost skin layer. Our results show that these extensions enable fully-automatic, generalized pre-processing of CT scans for further simulation of both skin dose and corresponding X-ray projection
Keywords: X-ray simulation; Computed tomography; Data preparation

Helical Dark-field Fiber Reconstruction - Felsner, Lina; Syben-Leisner, Christopher; Maier, Andreas; Riess, Christian (International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully3D), 2021)

Image Quality Assessment for Rigid Motion Compensation - Preuhs, Alexander; Manhart, Michael; Roser, Philipp; Stimpel, Bernhard; Syben-Leisner, Christopher; Psychogios, Marios; Kowarschik, Markus; Maier, Andreas (MedNeurips, 2019)

Joint Calibration and Motion Estimation in Weight-Bearing Cone-Beam CT of the Knee Joint using Fiducial Markers - Syben-Leisner, Christopher; Bier, Bastian; Berger, Martin; Aichert, André; Fahrig, Rebecca; Gold, Garry; Levenston, Marc; Maier, Andreas (Proceedings of the 2017 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2017)
UnivIS-Import:2017-12-18:Pub.2017.tech.IMMD.IMMD5.jointc_3
Keywords: C-arm CBCT; Image Reconstruction; Knee-Joint Imaging; Motion Estimation; Self-Calibration

Known Operator Learning enables Constrained Projection Geometry Conversion: Parallel to Cone-beam for Hybrid MR/X-ray Imaging - Syben-Leisner, Christopher; Stimpel, Bernhard; Roser, Philipp; Dörfler, Arnd; Maier, Andreas (IEEE Transactions on Medical Imaging, 2020)
X-ray imaging is a wide-spread real-time imaging technique. Magnetic Resonance Imaging (MRI) offers a multitude of contrasts that offer improved guidance to interventionalists. As such simultaneous real-time acquisition and overlay would be highly favorable for image-guided interventions, e.g., in stroke therapy. One major obstacle in this setting is the fundamentally different acquisition geometry. MRI k-space sampling is associated with parallel projection geometry, while the Xray acquisition results in perspective distorted projections. The classical rebinning methods to overcome this limitation inherently suffers from a loss of resolution. To counter this problem, we present a novel rebinning algorithm for parallel to cone-beam conversion. We derive a rebinning formula that is then used to find an appropriate deep neural network architecture. Following the known operator learning paradigm, the novel algorithm is mapped to a neural network with differentiable projection operators enabling data-driven learning of the remaining unknown operators. The evaluation aims in two directions: First, we give a profound analysis of the different hypotheses to the unknown operator and investigate the influence of numerical training data. Second, we evaluate the performance of the proposed method against the classical rebinning approach. We demonstrate that the derived network achieves better results than the baseline method and that such operators can be trained with simulated data without losing their generality making them applicable to real data without the need for retraining or transfer learning.<br
Keywords: Detectors; Magnetic resonance imaging; Geometry; X-ray imaging; Tomography; Image reconstruction

Known Operator Learning for a Hybrid Magnetic Resonance/X-ray Imaging Acquisition Scheme - Syben, Christopher (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2021)
With today’s technology, various non-invasive imaging methods provide detailed insight into the patient’s anatomy and support the physician during surgery. Depending on the signal type of each modality, the information collected is limited to narrow parts of the overall available information. A combination of several modalities makes it possible to increase the amount of information simultaneously made available to the physician. Especially a possible combination of the two most frequently used modalities, namely magnetic resonance imaging (MRI) and X-ray/computed tomography (CT) is of particular interest. Apart from the physical challenges, such a combination also poses a challenge to the acquisition scheme for a meaningful simultaneous acquisition with both modalities. This is especially true for the interventional environment, which implies additional requirements such as the real-time capability of the acquisition scheme. Such a combination, comes along with limitations for the signal recording schemes, up to the degree that an analytically correct solution can no longer be derived. Recently, there have been different attempts to overcome such limitations with machine learning. However, the solutions found are only partially comprehensible, and signal authenticity cannot be guaranteed. With the integration of prior knowledge about signal acquisition processes, such models can also be used in the medical field. In this thesis, we outline an acquisition scheme for a novel hybrid magnetic resonance (MR)/X-ray imaging system for the interventional environment and investigate the realization of parts of the scheme. Further, we investigate the concept of known operator learning, derive an implementation for operators in CT, and utilize the concept to enable the aforementioned hybrid MR/X-ray acquisition scheme. First, we investigate the possibilities and limitations for incorporating prior knowledge about the signal processing chain into machine learning algorithms. Using the universal approximation theorem (UAT), the benefits of mixing prior knowledge with neural networks are theoretically analyzed. Such a learning pipeline enriched with known operators allows the reduction of trainable parameters, and constrains the pipeline such that signal authenticity can be met, while the approximative power of deep learning (DL) can be utilized. In the course of the thesis, the concept is implemented for the CT reconstruction and evaluated from a methodological and algorithmic point of view. The results show that CT operators can be integrated into neural networks allowing gradient flow through the whole pipeline. The experiments suggest that such mixed pipelines can be trained only with numerical data if designed specifically for the problem. Furthermore, an open-source software framework called PYRO-NN is developed to make these benefits accessible to a broader community. Second, we develop a novel hybrid MR/X-ray acquisition scheme for image-guided interventions. The scheme acknowledges the high contrast diversity of MRI and the benefits of fast, high-resolution X-ray imaging. The information provided by both modalities is captured in different domains, making a subsequent exploitation of the complementary signals difficult, and is identified as a major obstacle to fully benefit from the simultaneous hybrid acquisition. The proposed scheme results in multiple orthographic MR projections and, in one perspectively distorted X-ray projection per frame. In the course of the thesis, we first develop a novel tomographic conversion scheme to overcome the limitations of classical geometrical rebinning. The tomographic rebinning is derived and performed utilizing the known operator learning scheme to learn an efficient convolution-based algorithm for the conversion. The novel concept is first designed and comprehensively evaluated on the 1-D/2-D case against the baseline method. Subsequently, the tomographic rebinning concept is developed for the clinically relevant 2-D/3-D case. In the course of the experiments, we can show that the efficient convolution-based algorithm can be learned with purely numeric training data. The presented method overcomes the limitation of the baseline method and provides sharper and more distinct projection than the baseline method. Hence, the learned tomographic rebinning algorithm is promising for the proposed hybrid MR/X-ray acquisition scheme, and the underlying known operator learning concept encourages further integration of the acquisition and contrast parameters into the trainable pipeline. In general, the benefit of incorporating domain knowledge into neural networks is highly promising and is possible for various other tasks even beyond medical imaging.<br
Keywords: CT; Machine Learning; Known Operator Learaning; Hybrid Imaging

Learning with Known Operators reduces Maximum Training Error Bounds. - Maier, Andreas; Syben-Leisner, Christopher; Stimpel, Bernhard; Würfl, Tobias; Hoffmann, Mathis; Schebesch, Frank; Fu, Weilin; Mill, Leonid; Kling, Lasse; Christiansen, Silke H. (Nature Machine Intelligence, 2019)
We describe an approach for incorporating prior knowledge into machine learning algorithms. We aim at applications in physics and signal processing in which we know that certain operations must be embedded into the algorithm. Any operation that allows computation of a gradient or sub-gradient towards its inputs is suited for our framework. We derive a maximal error bound for deep nets that demonstrates that inclusion of prior knowledge results in its reduction. Furthermore, we also show experimentally that known operators reduce the number of free parameters. We apply this approach to various tasks ranging from CT image reconstruction over vessel segmentation to the derivation of previously unknown imaging algorithms. As such the concept is widely applicable for many researchers in physics, imaging, and signal processing. We assume that our analysis will support further investigation of known operators in other fields of physics, imaging, and signal processing.

Magnetic Resonance Fingerprinting Reconstruction Using Recurrent Neural Networks - Hoppe, Elisabeth; Thamm, Florian; Körzdörfer, Gregor; Syben-Leisner, Christopher; Schirrmacher, Franziska; Nittka, Mathias; Pfeuffer, Josef; Meyer, Heiko; Maier, Andreas (German Medical Data Sciences: Shaping Change – Creative Solutions for Innovative Medicine, 2019)
Magnetic Resonance Fingerprinting (MRF) is an imaging technique acquiring unique time signals for different tissues. Although the acquisition is highly accelerated, the reconstruction time remains a problem, as the state-of-the-art template matching compares every signal with a set of possible signals. To overcome this limitation, deep learning based approaches, e.g. Convolutional Neural Networks (CNNs) have been proposed. In this work, we investigate the applicability of Recurrent Neural Networks (RNNs) for this reconstruction problem, as the signals are correlated in time. Compared to previous methods based on CNNs, RNN models yield significantly improved results using in-vivo data.
CRIS-Team Scopus Importer:2019-09-24
Keywords: Artificial Neural Networks; Magnetic Resonance Fingerprinting; Magnetic Resonance Fingerprinting Reconstruction; Recurrent Neural Networks

Maximum Likelihood Estimation of Head Motion Using Epipolar Consistency - Preuhs, Alexander; Ravikumar, Nishant; Manhart, Michael; Stimpel, Bernhard; Hoppe, Elisabeth; Syben-Leisner, Christopher; Kowarschik, Markus; Maier, Andreas (Informatik aktuell, 2019)
Open gantry C-arm systems that are placed within the interventional room enable 3-D imaging and guidance for stroke therapy without patient transfer. This can profit in drastically reduced time-totherapy, however, due to the interventional setting, the data acquisition is comparatively slow. Thus, involuntary patient motion needs to be estimated and compensated to achieve high image quality. Patient motion results in a misalignment of the geometry and the acquired image data. Consistency measures can be used to restore the correct mapping to compensate the motion. They describe constraints on an idealized imaging process which makes them also sensitive to beam hardening, scatter, truncation or overexposure. We propose a probabilistic approach based on the Student’s t-distribution to model image artifacts that affect the consistency measure without sourcing from motion.
CRIS-Team Scopus Importer:2019-05-14

MR-projection imaging with perspective distortion as in X-ray fluoroscopy for interventional X/MR-hybrid applications - Lommen, Jonathan; Syben-Leisner, Christopher; Stimpel, Bernhard; Bayer, Siming; Nagel, Armin Michael; Fahrig, Rebecca; Dörfler, Arnd; Maier, Andreas (MR-projection imaging with perspective distortion as in X-ray fluoroscopy for interventional X/MR-hybrid applications, 2018)

MR to X-ray Projection Image Synthesis - Stimpel, Bernhard; Syben-Leisner, Christopher; Würfl, Tobias; Mentl, Katrin; Dörfler, Arnd; Maier, Andreas (Proceedings of the 5th International Conference on Image Formation in X-ray Computed Tomography (CT-Meeting), 2018)
UnivIS-Import:2018-09-06:Pub.2018.tech.IMMD.IMMD5.mrtoxr_0

Multi-Channel Volumetric Neural Network for Knee Cartilage Segmentation in Cone-Beam CT - Maier, Jennifer; Rivera Monroy, Luis Carlos; Syben-Leisner, Christopher; Jeon, Yejin; Choi, Jang-Hwan; Hall, Mary; Levenston, Marc; Gold, Garry; Fahrig, Rebecca; Maier, Andreas (Bildverarbeitung für die Medizin 2020, 2020)
Analyzing knee cartilage thickness and strain under load can help to further the understanding of the effects of diseases like Osteoarthritis. A precise segmentation of the cartilage is a necessary prerequisite for this analysis. This segmentation task has mainly been addressed in Magnetic Resonance Imaging, and was rarely investigated on contrast-enhanced Computed Tomography, where contrast agent visualizes the border between femoral and tibial cartilage. To overcome the main drawback of manual segmentation, namely its high time investment, we propose to use a 3D Convolutional Neural Network for this task. The presented architecture consists of a V-Net with SeLu activation, and a Tversky loss function. Due to the high imbalance between very few cartilage pixels and many background pixels, a high false positive rate is to be expected. To reduce this rate, the two largest segmented point clouds are extracted using a connected component analysis, since they most likely represent the medial and lateral tibial cartilage surfaces. The resulting segmentations are compared to manual segmentations, and achieve on average a recall of 0.69, which confirms the feasibility of this approach.<br

Multi-Modal Deep Guided Filtering for Comprehensible Medical Image Processing Supplementary Material - Stimpel, Bernhard; Syben-Leisner, Christopher; Schirrmacher, Franziska; Hölter, Philip; Dörfler, Arnd; Maier, Andreas (IEEE Transactions on Medical Imaging, 2019)
EVALuna2:212806

Multi-Modal Super-Resolution with Deep Guided Filtering - Stimpel, Bernhard; Syben-Leisner, Christopher; Schirrmacher, Franziska; Dörfler, Arnd; Maier, Andreas; Hölter, Philip (Informatik aktuell, 2019)
Despite the visually appealing results, most Deep Learningbased super-resolution approaches lack the comprehensibility that is required for medical applications. We propose a modified version of the locally linear guided filter for the application of super-resolution in medical imaging. The guidance map itself is learned end-to-end from multimodal inputs, while the actual data is only processed with known operators. This ensures comprehensibility of the results and simplifies the implementation of guarantees. We demonstrate the possibilities of our approach based on multi-modal MR and cross-modal CT and MR data. For both datasets, our approach performs clearly better than bicubic upsampling. For projection images, we achieve SSIMs of up to 0.99, while slice image data results in SSIMs of up to 0.98 for four-fold upsampling given an image of the respective other modality at full resolution. In addition, end-to-end learning of the guidance map considerably improves the quality of the results.
CRIS-Team Scopus Importer:2019-05-14

On the Influence of Acquisition Angle and Slice Thickness on the in-plane Spatial Resolution of Calcifications in Digital Breast Tomosynthesis - Luckner, Christoph; Schebesch, Frank; Syben-Leisner, Christopher; Mertelmeier, Thomas; Maier, Andreas; Ritschl, Ludwig (Proceedings of the Fifth International Conference on Image Formation in X-Ray Computed Tomography, 2018)
UnivIS-Import:2018-09-06:Pub.2018.tech.IMMD.IMMD5.onthei_5

Optimizing CT Scan Geometries With and Without Gradients - Thies, Mareike; Wagner, Fabian; Maul, Noah; Pfaff, Laura; Schneider, Linda-Sophie; Syben, Christopher; Maier, Andreas (17th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully3D), 2023)
In computed tomography (CT), the projection geometry used for data acquisition needs to be known precisely to obtain a clear reconstructed image. Rigid patient motion is a cause for misalignment between measured data and employed geometry. Commonly, such motion is compensated by solving an optimization problem that, e.g., maximizes the quality of the reconstructed image with respect to the projection geometry. So far, gradient-free optimization algorithms have been utilized to find the solution for this problem. Here, we show that gradient-based optimization algorithms are a possible alternative and compare the performance to their gradient-free counterparts on a benchmark motion compensation problem. Gradient-based algorithms converge substantially faster while being comparable to gradient-free algorithms in terms of capture range and robustness to the number of free parameters. Hence, gradient-based optimization is a viable alternative for the given type of problems.<br

Precision Learning: Reconstruction Filter Kernel Discretization - Syben-Leisner, Christopher; Stimpel, Bernhard; Breininger, Katharina; Würfl, Tobias; Fahrig, Rebecca; Dörfler, Arnd; Maier, Andreas (Proceedings of the 5th International Conference on Image Formation in X-ray Computed Tomography, 2018)
UnivIS-Import:2018-09-11:Pub.2018.tech.IMMD.IMMD5.precis_0

Precision Learning: Towards Use of Known Operators in Neural Networks - Maier, Andreas; Schebesch, Frank; Syben-Leisner, Christopher; Würfl, Tobias; Steidl, Stefan; Choi, Jang-Hwan; Fahrig, Rebecca (2018 24rd International Conference on Pattern Recognition (ICPR), 2018)
UnivIS-Import:2018-09-06:Pub.2018.tech.IMMD.IMMD5.precis_5

Projection image-to-image translation in hybrid x-ray/MR imaging - Stimpel, Bernhard; Syben-Leisner, Christopher; Würfl, Tobias; Breininger, Katharina; Lommen, Jonathan; Dörfler, Arnd; Maier, Andreas (Progress in Biomedical Optics and Imaging - Proceedings of SPIE, 2019)
The potential benefit of hybrid X-ray and MR imaging in the interventional environment is large due to the combination of fast imaging with high contrast variety. However, a vast amount of existing image enhancement methods requires the image information of both modalities to be present in the same domain. To unlock this potential, we present a solution to image-to-image translation from MR projections to corresponding x-ray projection images. The approach is based on a state-of-the-art image generator network that is modified to fit the specific application. Furthermore, we propose the inclusion of a gradient map in the loss function to allow the network to emphasize high-frequency details in image generation. Our approach is capable of creating x-ray projection images with natural appearance. Additionally, our extensions show clear improvement compared to the baseline method.
CRIS-Team Scopus Importer:2019-07-12

Projection-to-Projection Translation for Hybrid X-ray and Magnetic Resonance Imaging - Stimpel, Bernhard; Syben-Leisner, Christopher; Würfl, Tobias; Breininger, Katharina; Hölter, Philip; Dörfler, Arnd; Maier, Andreas (Scientific Reports, 2019)
Hybrid X-ray and magnetic resonance (MR) imaging promises large potential in interventional medical imaging applications due to the broad variety of contrast of MRI combined with fast imaging of X-ray-based modalities. To fully utilize the potential of the vast amount of existing image enhancement techniques, the corresponding information from both modalities must be present in the same domain. For image-guided interventional procedures, X-ray fluoroscopy has proven to be the modality of choice. Synthesizing one modality from another in this case is an ill-posed problem due to ambiguous signal and overlapping structures in projective geometry. To take on these challenges, we present a learning-based solution to MR to X-ray projection-to-projection translation. We propose an image generator network that focuses on high representation capacity in higher resolution layers to allow for accurate synthesis of fine details in the projection images. Additionally, a weighting scheme in the loss computation that favors high-frequency structures is proposed to focus on the important details and contours in projection imaging. The proposed extensions prove valuable in generating X-ray projection images with natural appearance. Our approach achieves a deviation from the ground truth of only 6% and structural similarity measure of 0.913 ± 0.005. In particular the high frequency weighting assists in generating projection images with sharp appearance and reduces erroneously synthesized fine details.<br

Reconstruction of Voxels with Position- and Angle-Dependent Weightings - Felsner, Lina; Würfl, Tobias; Syben-Leisner, Christopher; Roser, Philipp; Preuhs, Alexander; Maier, Andreas; Riess, Christian (The 6th International Conference on Image Formation in X-Ray Computed Tomography, 2020)

RinQ Fingerprinting: Recurrence-Informed Quantile Networks for Magnetic Resonance Fingerprinting - Hoppe, Elisabeth; Thamm, Florian; Koerzdoerfer, Gregor; Syben-Leisner, Christopher; Nittka, Mathias; Pfeuffer, Josef; Meyer, Heiko; Maier, Andreas (Bilderverarbeitung für die Medizin Algorithmen - Systeme - Anwendungen, 2020)

RinQ Fingerprinting: Recurrence-Informed Quantile Networks for Magnetic Resonance Fingerprinting - Hoppe, Elisabeth; Thamm, Florian; Körzdörfer, Gregor; Syben-Leisner, Christopher; Schirrmacher, Franziska; Nittka, Mathias; Pfeuffer, Josef; Meyer, Heiko; Maier, Andreas (Proceedings of MICCAI 2019, 2019)
Recently, Magnetic Resonance Fingerprinting (MRF) was proposed as a quantitative imaging technique for the simultaneous acquisition of tissue parameters such as relaxation times T1 and T2. Although the acquisition is highly accelerated, the state-of-the-art reconstruction suffers from long computation times: Template matching methods are used to find the most similar signal to the measured one by comparing it to pre-simulated signals of possible parameter combinations in a discretized dictionary. Deep learning approaches can overcome this limitation, by providing the direct mapping from the measured signal to the underlying parameters by one forward pass through a network. In this work, we propose a Recurrent Neural Network (RNN) architecture in combination with a novel quantile layer. RNNs are well suited for the processing of time-dependent signals and the quantile layer helps to overcome the noisy outliers by considering the spatial neighbors of the signal. We evaluate our approach using in-vivo data from multiple brain slices and several volunteers, running various experiments. We show that the RNN approach with small patches of complex-valued input signals in combination with a quantile layer outperforms other architectures, e.g. previously proposed CNNs for the MRF reconstruction reducing the error in T1 and T2 by more than 80%.<br

Self-Calibration and Simultaneous Motion Estimation for C-arm CT using Fiducial Markers - Syben-Leisner, Christopher; Bier, Bastian; Berger, Martin; Aichert, André; Fahrig, Rebecca; Gold, Garry; Levenston, Marc; Maier, Andreas (Bildverarbeitung für die Medizin 2017: Algorithmen - Systeme - Anwendungen., 2017)
C-arm cone-beam CT systems have an increasing popularity in the clinical environment due to their highly flexible scan trajectories. Recent work used these systems to acquire images of the knee joint under weight-bearing conditions. During the scan, the patient is in a standing or in a squatting position and is likely to show involuntary motion, which corrupts image reconstruction. The state-of-the-art fully automatic motion compensation relies on fiducial markers for motion estimation. Due to the not reproducible horizontal trajectory, the system has to be calibrated with a calibration phantom before or after each scan. In this work we present a method to incorporate a self-calibration into the existing motion compensation framework without the need of prior geometric calibration. Quantitative and qualitative evaluations on a numerical phantom as well as clinical data, show superior results compared to the current state-of-the-art method. Moreover, the clinical workflow is improved, as a dedicated system calibration for weight-bearing acquisitions is no longer required.
UnivIS-Import:2018-09-06:Pub.2017.tech.IMMD.IMMD5.selfca_1

Simultaneous Estimation of X-Ray Back-Scatter and Forward-Scatter Using Multi-task Learning - Roser, Philipp; Zhong, Xia; Birkhold, Annette; Preuhs, Alexander; Syben-Leisner, Christopher; Hoppe, Elisabeth; Strobel, Norbert; Kowarschik, Markus; Fahrig, Rebecca; Maier, Andreas (Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, 2020)
Scattered radiation is a major concern impacting X-ray image-guided procedures in two ways. First, back-scatter significantly contributes to patient (skin) dose during complicated interventions. Second, forward-scattered radiation reduces contrast in projection images and introduces artifacts in 3-D reconstructions. While conventionally employed anti-scatter grids improve image quality by blocking X-rays, the additional attenuation due to the anti-scatter grid at the detector needs to be compensated for by a higher patient entrance dose. This also increases the room dose affecting the staff caring for the patient. For skin dose quantification, back-scatter is usually accounted for by applying pre-determined scalar back-scatter factors or linear point spread functions to a primary kerma forward projection onto a patient surface point. However, as patients come in different shapes, the generalization of conventional methods is limited. Here, we propose a novel approach combining conventional techniques with learning-based methods to simultaneously estimate the forward-scatter reaching the detector as well as the back-scatter affecting the patient skin dose. Knowing the forward-scatter, we can correct X-ray projections, while a good estimate of the back-scatter component facilitates an improved skin dose assessment. To simultaneously estimate forward-scatter as well as back-scatter, we propose a multi-task approach for joint back- and forward-scatter estimation by combining X-ray physics with neural networks. We show that, in theory, highly accurate scatter estimation in both cases is possible. In addition, we identify research directions for our multi-task framework and learning-based scatter estimation in general.<br
Keywords: X-ray scatter Skin dose Multi-task learning

Technical Note: PYRO-NN: Python reconstruction operators in neural networks - Syben-Leisner, Christopher; Michen, Markus; Stimpel, Bernhard; Seitz, Stephan; Ploner, Stefan; Maier, Andreas (Medical Physics, 2019)
Purpose: Recently, several attempts were conducted to transfer deep learning to medical image reconstruction. An increasingly number of publications follow the concept of embedding the computed tomography (CT) reconstruction as a known operator into a neural network. However, most of the approaches presented lack an efficient CT reconstruction framework fully integrated into deep learning environments. As a result, many approaches use workarounds for mathematically unambiguously solvable problems. Methods: PYRO-NN is a generalized framework to embed known operators into the prevalent deep learning framework Tensorflow. The current status includes state-of-the-art parallel-, fan-, and cone-beam projectors, and back-projectors accelerated with CUDA provided as Tensorflow layers. On top, the framework provides a high-level Python API to conduct FBP and iterative reconstruction experiments with data from real CT systems. Results: The framework provides all necessary algorithms and tools to design end-to-end neural network pipelines with integrated CT reconstruction algorithms. The high-level Python API allows a simple use of the layers as known from Tensorflow. All algorithms and tools are referenced to a scientific publication and are compared to existing non-deep learning reconstruction frameworks. To demonstrate the capabilities of the layers, the framework comes with baseline experiments, which are described in the supplementary material. The framework is available as open-source software under the Apache 2.0 licence at https://github.com/csyben/PYRO-NN. Conclusions: PYRO-NN comes with the prevalent deep learning framework Tensorflow and allows to setup end-to-end trainable neural networks in the medical image reconstruction context. We believe that the framework will be a step toward reproducible research and give the medical physics community a toolkit to elevate medical image reconstruction with new deep learning techniques.
CRIS-Team Scopus Importer:2019-10-22
Keywords: inverse problems; known operator learning; machine learning; open source; reconstruction

Towards in-vivo X-ray nanoscopy: The effect of motion on image quality - Mill, Leonid; Bier, Bastian; Syben-Leisner, Christopher; Kling, Lasse; Grüneboom, Anika; Christiansen, Silke; Schett, Georg; Maier, Andreas (Informatik aktuell, 2018)
Novel X-Ray Microscopy (XRM) systems allow to study the internal structure of a specimen on nanoscale. A possible use of this non-destructive technology is motivated in the medical research area. In-Vivo investigation of medication over a period of time and its effects on perfusion and bony structure might lead to a better understanding of drug mechanisms and diseases like Osteoporosis and could lead to new approaches to their treatment. The first step towards in-vivo XRM imaging is to investigate the suitability of recent XRM systems for this task and subsequently to determine the system parameters. In this context, the impact of mice motion on the image quality is studied in this work. This paper aims to simulate the effects of breathing motion and muscle relaxation of the mice on the reconstructed images, which already effects the projection images. We therefore assume a mouse’s respiration motion pattern, which happens four time during a single projection acquisitions, and the muscle relaxation movement due to anesthesia and simulate its impacts on image quality. Additionally, we show that a frame rate of at least 16 fps is needed to capture in-vivo movements in order to apply state-of-the-art motion correction methods.<br

User Loss A Forced-Choice-Inspired Approach to Train Neural Networks Directly by User Interaction - Zarei, Shahab; Stimpel, Bernhard; Syben-Leisner, Christopher; Maier, Andreas (Informatik aktuell, 2019)
In this paper, we investigate whether is it possible to train a neural network directly from user inputs. We consider this approach to be highly relevant for applications in which the point of optimality is not well-defined and user-dependent. Our application is medical image denoising which is essential in fluoroscopy imaging. In this field every user, i.e. physician, has a different flavor and image quality needs to be tailored towards each individual. To address this important problem, we propose to construct a loss function derived from a forced-choice experiment. In order to make the learning problem feasible, we operate in the domain of precision learning, i.e., we inspire the network architecture by traditional signal processing methods in order to reduce the number of trainable parameters. The algorithm that was used for this is a Laplacian pyramid with only six trainable parameters. In the experimental results, we demonstrate that two image experts who prefer different filter characteristics between sharpness and de-noising can be created using our approach. Also models trained for a specific user perform best on this users test data. This approach opens the way towards implementation of direct user feedback in deep learning and is applicable for a wide range of application.
CRIS-Team Scopus Importer:2019-05-14

X-ray Scatter Estimation Using Deep Splines - Roser, Philipp; Birkhold, Annette; Preuhs, Alexander; Syben-Leisner, Christopher; Felsner, Lina; Hoppe, Elisabeth; Strobel, Norbert; Kowarschik, Markus; Fahrig, Rebecca; Maier, Andreas (IEEE Transactions on Medical Imaging, 2021)
X-ray scatter compensation is a very desirable technique in flat-panel X-ray imaging and cone-beam computed tomography. State-of-the-art U-net based scatter removal approaches yielded promising results. However, as there are no physics’ constraints applied to the output of the U-Net, it cannot be ruled out that it yields spurious results. Unfortunately, in the context of medical imaging, those may be misleading and could lead to wrong conclusions. To overcome this problem, we propose to embed B-splines as a known operator into neural networks. This inherently constrains their predictions to well-behaved and smooth functions. In a study using synthetic head and thorax data as well as real thorax phantom data, we found that our approach performed on par with U-net when comparing both algorithms based on quantitative performance metrics. However, our approach not only reduces runtime and parameter complexity, but we also found it much more robust to unseen noise levels. While the U-net responded with visible artifacts, the proposed approach preserved the X-ray signal’s frequency characteristics.
CRIS-Team Scopus Importer:2021-05-07
Keywords: Approximation; B-spline; Detectors; Estimation; Neural network; Neural networks; Photonics; Splines (mathematics); Thorax; X-ray imaging; X-ray scatter