Abstract: Leveraging Image Captions for Selective Whole Slide Image Annotation - Qiu, Jingna; Aubreville, Marc; Wilm, Frauke; Öttl, Mathias; Utz, Jonas; Schlereth, Maja; Breininger, Katharina (Bildverarbeitung für die Medizin 2025, 2025)
Obtaining dense annotations for histopathological whole-slide images (WSI), such as segmentation masks or mitotic figure identification, is a labor intensive process due to the large image size and the extensive manual effort required for annotation. Identifying informative regions in WSIs for annotation while leaving other regions unlabeled can significantly reduce the annotation effort. These selected annotation regions should contain valuable training information that allows proper model training without significantly impacting performance compared to full annotatio

Adaptive Region Selection for Active Learning in Whole Slide Image Semantic Segmentation - Qiu, Jingna; Wilm, Frauke; Öttl, Mathias; Schlereth, Maja; Liu, Chang; Heimann, Tobias; Aubreville, Marc; Breininger, Katharina (The 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023), 2023)
The process of annotating histological gigapixel-sized whole slide images (WSIs) at the pixel level for the purpose of training a supervised segmentation model is time-consuming. Region-based active learning (AL) involves training the model on a limited number of annotated image regions instead of requesting annotations of the entire images. These annotation regions are iteratively selected, with the goal of optimizing model performance while minimizing the annotated area. The standard method for region selection evaluates the informativeness of all square regions of a specified size and then selects a specific quantity of the most informative regions. We find that the efficiency of this method highly depends on the choice of AL step size (i.e., the combination of region size and the number of selected regions per WSI), and a suboptimal AL step size can result in redundant annotation requests or inflated computation costs. This paper introduces a novel technique for selecting annotation regions adaptively, mitigating the reliance on this AL hyperparameter. Specifically, we dynamically determine each region by first identifying an informative area and then detecting its optimal bounding box, as opposed to selecting regions of a uniform predefined shape and size as in the standard method. We evaluate our method using the task of breast cancer metastases segmentation on the public CAMELYON16 dataset and show that it consistently achieves higher sampling efficiency than the standard method across various AL step sizes. With only 2.6% of tissue area annotated, we achieve full annotation performance and thereby substantially reduce the costs of annotating a WSI dataset. The source code is available at https://github.com/DeepMicroscopy/AdaptiveRegionSelection.
Keywords: Active learning; Region selection; Whole slide images.

Advanced neural networks for classification of MRI in psoriatic arthritis, seronegative, and seropositive rheumatoid arthritis - Folle, Lukas; Bayat, Sara; Kleyer, Arnd; Fagni, Filippo; Kapsner, Lorenz; Schlereth, Maja; Meinderink, Timo; Breininger, Katharina; Tascilar, Koray; Krönke, Gerhard; Uder, Michael; Sticherling, Michael; et al. (Rheumatology, 2022)
Objectives: To evaluate whether neural networks can distinguish between seropositive rheumatoid arthritis (RA), seronegative RA and psoriatic arthritis (PsA) based on inflammatory patterns from hand MRI and to test how psoriasis patients with subclinical inflammation fit into such patterns. Methods: ResNet neural networks were utilized to compare (i) seropositive RA vs. PsA, (ii) seronegative RA vs. PsA and (iii) seropositive vs. seronegative RA with respect to hand MRI data. Results from T1 coronal, T2 coronal, T1 coronal and axial fat suppressed contrast-enhanced (CE) and T2 fat suppressed axial sequences were used. The performance of such trained networks was analyzed by the area-under-the-receiver-operating-characteristic curve (AUROC) with and without presentation of demographic and clinical parameters. Additionally, the trained networks were applied to psoriasis patients without clinical arthritis. Results: MRI scans from 649 patients (135 seronegative RA, 190 seropositive RA, 177 PsA, 147 psoriasis) were fed into ResNet neural networks. AUROC was 75% for seropositive RA vs. PsA, 74% for seronegative RA vs. PsA and 67% for seropositive vs. seronegative RA. All MRI sequences were relevant for classification, however, when deleting contrast agent-based sequences the loss of performance was only marginal. The addition of demographic and clinical data to the networks did not provide significant improvements for classification. Psoriasis patients were mostly assigned to PsA by the neural networks, suggesting that a PsA-like MRI pattern may be present early in the course of psoriatic disease. Conclusion: Neural networks can be successfully trained to distinguish MRI inflammation related to seropositive RA, seronegative RA, and PsA.
Keywords: Rheumatoid arthritis; Psoriatic arthritis; neural networks; deep learning

Automatic Classification of Neuromuscular Diseases in Children Using Photoacoustic Imaging - Schlereth, Maja; Stromer, Daniel; Breininger, Katharina; Wagner, Alexandra; Tan, Lina; Maier, Andreas; Knieling, Ferdinand (Informatik aktuell, 2022)
Neuromuscular diseases (NMDs) cause a significant burden for both healthcare systems and society. They can lead to severe progressive muscle weakness, muscle degeneration, contracture, deformity and progressive disability. The NMDs evaluated in this study often manifest in early childhood. As subtypes of disease, e.g. Duchenne muscular dystropy (DMD) and spinal muscular atrophy (SMA), are difficult to differentiate at the beginning and worsen quickly, fast and reliable differential diagnosis is crucial. Photoacoustic and ultrasound imaging has shown great potential to visualize and quantify the extent of different diseases. The addition of automatic classification of such image data could further improve standard diagnostic procedures.We compare deep learning-based 2-class and 3-class classifiers based on VGG16 for differentiating healthy from diseased muscular tissue. This work shows promising results with high accuracies above 0.86 for the 3-class problem and can be used as a proof of concept for future approaches for earlier diagnosis and therapeutic monitoring of NMDs.<br
CRIS-Team Scopus Importer:2022-05-06

Automatic Muscle Segmentation for the Diagnosis of Peripheral Artery Disease Using Multispectral Optoacoustic Tomography - Schillinger, Moritz; Schlereth, Maja; Boraud, Gaspard; Zahnd, Guillaume; Dehner, Christoph; Li, Yi; Kempf, Julius; Caranovic, Milenko; Haney, Briain; Rother, Ulrich; Breininger, Katharina (Progress in Biomedical Optics and Imaging - Proceedings of SPIE, 2025)
Multispectral optoacoustic tomography (MSOT) is a medical imaging modality capable of visualizing chromophore concentrations, such as oxygenated (HbO2) and deoxygenated hemoglobin (Hb), making it particularly useful for diagnosing blood-perfusion-related diseases like peripheral artery disease (PAD). Previous MSOT-based diagnostic studies involved experts manually selecting a region of interest (ROI) with a predefined shape in the target muscle to analyze blood oxygenation. This study aims to automate this process using a deep-learning-based approach. We present a pipeline that uses co-registered ultrasound images to automatically place an ROI necessary for PAD diagnosis in the MSOT image by training a deep-learning-based segmentation model to locate the target muscle. The resulting segmentation mask is overlaid onto the corresponding optoacoustic image to extract mean HbO2 and Hb intensities within the ROIs. We evaluated our pipeline on two separate PAD-related MSOT-datasets. When comparing blood oxygenation in the automatically generated ROIs on the MSOT images to identify PAD patients, we achieved areas under the ROC curve (AUCs) of 0.87 (vs. 0.85 for manually drawn ROIs) and 0.76 (vs. 0.77 for manually drawn ROIs) on the respective datasets. Our results indicate that optoacoustic ROI placement using deep-learning-based ultrasound segmentation is feasible and performs comparably to manually drawn ROIs by clinical experts. This approach could reduce the annotation effort in future MSOT studies and provide an ROI with greater physiological relevance compared to the arbitrary shapes used in related MSOT research. Future work could extend this approach to other body sites and diseases.
CRIS-Team Scopus Importer:2025-05-16

CLASSIFICATION OF PSORIATIC ARTHRITIS, SERONEGATIVE RHEUMATOID ARTHRITIS, AND SEROPOSITIVE RHEUMATOID ARTHRITIS USING DEEP LEARNING ON MAGNETIC RESONANCE IMAGING - Folle, Lukas; Bayat, Sara; Kleyer, Arnd; Fagni, Filippo; Kapsner, Lorenz; Schlereth, Maja; Meinderink, Timo; Breininger, Katharina; Tascilar, Koray; Krönke, Gerhard; Uder, Michael; Sticherling, Michael; et al. (Supplement 1, 2022)
Background: While MRI evaluation of joints has been primarily used to quantify inflammation at a cross-sectional and longitudinal level, less is known about the potential of MRI in distinguishing different patterns of inflammation in the various forms of arthritis. Objectives: To evaluate (i) whether deep learning using neural networks can be trained to distinguish between seropositive rheumatoid arthritis (RA+), seronegative RA (RA-), and psoriatic arthritis (PsA) based on structural inflammatory patterns on hand magnetic resonance imaging and (ii) to assess if psoriasis patients with subclinical inflammation fit into such patterns. Methods: ResNet 3D [1] neural networks were trained to distinguish (i) RA+ vs. PsA, (ii) RA- vs. PsA and (iii) RA+ vs. RA- with respect to hand MRI data. Diagnosis of patients was determined using the following guidelines: ACR/EULAR 2010 [2] for RA and CASPAR [3] for PsA. Results from T1 coronal, T2 coronal, T1 coronal and axial fat suppressed contrast-enhanced (CE) and T2 fat suppressed axial sequences were used. The performance of such trained networks was analyzed by the area-under-the-receiver-operating-characteristic curve (AUROC) with and without imputation of demographic and clinical parameters ( Figure 1A ). Additionally, the trained networks were applied to psoriasis patients without clinical signs of PsA. Results: MRI scans from 649 patients (135 RA-, 190 RA+, 177 PsA, 147 psoriasis) were included ( Table 1 ). The AUROC for differentiation between disease entities was 75% (SD 3%) for RA+ vs. PsA, 74% (SD 8%) for RA- vs. PsA, and 67% (6%) for RA+ vs. RA-. All MRI sequences were relevant for classification, however, when deleting CE sequences, the loss of performance was only marginal. The addition of patient-specific data to the networks did not provide significant improvements. Increasing amounts of training data demonstrated improved performance of the networks ( Figure 1B ). Psoriasis patients were mostly assigned to PsA by the neural networks, suggesting that PsA-like MRI pattern may be present early in the course of psoriatic disease. Conclusion: Deep learning can be successfully applied to differentiate MRI inflammatory patterns related to RA+, RA-, and PsA. Early changes in psoriasis patients can be recognized by neural networks and are characterized by a pattern that allowed the networks to classify them as PsA. <
Keywords: Rheumatoid arthritis; deep learning; neural networks

Deep learning-based classification of erosion, synovitis and osteitis in hand MRI of patients with inflammatory arthritis - Schlereth, Maja; Yalcin-Mutlu, Melek; Utz, Jonas; Bayat, Sara; Heimann, Tobias; Qiu, Jingna; Ehring, Chris; Liu, Chang; Uder, Michael; Kleyer, Arnd; Simon, David; Roemer, Frank; et al. (RMD Open, 2024)
OBJECTIVES: To train, test and validate the performance of a convolutional neural network (CNN)-based approach for the automated assessment of bone erosions, osteitis and synovitis in hand MRI of patients with inflammatory arthritis. METHODS: Hand MRIs (coronal T1-weighted, T2-weighted fat-suppressed, T1-weighted fat-suppressed contrast-enhanced) of rheumatoid arthritis (RA) and psoriatic arthritis (PsA) patients from the rheumatology department of the Erlangen University Hospital were assessed by two expert rheumatologists using the Outcome Measures in Rheumatology-validated RA MRI Scoring System and PsA MRI Scoring System scores and were used to train, validate and test CNNs to automatically score erosions, osteitis and synovitis. Scoring performance was compared with human annotations in terms of macro-area under the receiver operating characteristic curve (AUC) and balanced accuracy using fivefold cross-validation. Validation was performed on an independent dataset of MRIs from a second patient cohort. RESULTS: In total, 211 MRIs from 112 patients (14 906 region of interests (ROIs)) were included for training/internal validation using cross-validation and 220 MRIs from 75 patients (11 040 ROIs) for external validation of the networks. The networks achieved high mean (SD) macro-AUC of 92%±1% for erosions, 91%±2% for osteitis and 85%±2% for synovitis. Compared with human annotation, CNNs achieved a high mean Spearman correlation for erosions (90±2%), osteitis (78±8%) and synovitis (69±7%), which remained consistent in the validation dataset. CONCLUSIONS: We developed a CNN-based automated scoring system that allowed a rapid grading of erosions, osteitis and synovitis with good diagnostic accuracy and using less MRI sequences compared with conventional scoring. This CNN-based approach may help develop standardised cost-efficient and time-efficient assessments of hand MRIs for patients with arthritis.
CRIS-Team Scopus Importer:2024-06-28
Keywords: classification; machine learning; magnetic resonance imaging; psoriatic arthritis; rheumatoid arthritis

Focus on Content not Noise: Improving Image Generation for Nuclei Segmentation by Suppressing Steganography in CycleGAN - Utz, Jonas; Weise, Tobias; Schlereth, Maja; Wagner, Fabian; Thies, Mareike; Gu, Mingxuan; Uderhardt, Stefan; Breininger, Katharina (Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023, 2023)
Annotating nuclei in microscopy images for the training of neural networks is a laborious task that requires expert knowledge and suffers from inter- and intra-rater variability, especially in fluorescence microscopy. Generative networks such as CycleGAN can inverse the process and generate synthetic microscopy images for a given mask, thereby building a synthetic dataset. However, past works report content inconsistencies between the mask and generated image, partially due to CycleGAN minimizing its loss by hiding shortcut information for the image reconstruction in high frequencies rather than encoding the desired image content and learning the target task. In this work, we propose to remove the hidden shortcut information, called steganography, from generated images by employing a low pass filtering based on the discrete cosine transform (DCT). We show that this increases coherence between generated images and cycled masks and evaluate synthetic datasets on a downstream nuclei segmentation task. Here we achieve an improvement of 5.4 percentage points in the F1-score compared to a vanilla CycleGAN. Integrating advanced regularization techniques into the CycleGAN architecture may help mitigate steganography-related issues and produce more accurate synthetic datasets for nuclei segmentation.
CRIS-Team Scopus Importer:2024-12-16
Keywords: cycle gan; data generation; dct; fluorescence microscopy; nuclei; segmentation; steganography; synthetic data

Initial Investigations Towards Non-invasive Monitoring of Chronic Wound Healing Using Deep Learning and Ultrasound Imaging - Schlereth, Maja; Stromer, Daniel; Mantri, Yash; Tsujimoto, Jason; Breininger, Katharina; Maier, Andreas; Anderson, Caesar; Garimella, Pranav S.; Jokerst, Jesse V. (Informatik aktuell, 2022)
Chronic wounds including diabetic and arterial/venous insufficiency injuries have become a major burden for healthcare systems worldwide. Demographic changes suggest that wound care will play an even bigger role in the coming decades. Predicting and monitoring response to therapy in wound care is currently largely based on visual inspection with little information on the underlying tissue. Thus, there is an urgent unmet need for innovative approaches that facilitate personalized diagnostics and treatments at the point-of-care. It has been recently shown that ultrasound imaging can monitor response to therapy in wound care, but this work required onerous manual image annotations. In this study we present initial results of a deep learning-based automatic segmentation of cross-sectional wound size in ultrasound images and identify requirements and challenges for future research on this application. Evaluation of the segmentation results underscores the potential of the proposed deep learning approach to complement non-invasive imaging with Dice scores of 0.34 (U-Net, FCN) and 0.27 (ResNet-U-Net) but also highlights the need for improving robustness further.We conclude that deep learning-supported analysis of non-invasive ultrasound images is a promising area of research to automatically extract cross-sectional wound size and depth information with potential value in monitoring response to therapy.<br
CRIS-Team Scopus Importer:2022-05-06

Leveraging Image Captions for Selective Whole Slide Image Annotation - Qiu, Jingna; Aubreville, Marc; Wilm, Frauke; Öttl, Mathias; Utz, Jonas; Schlereth, Maja; Breininger, Katharina (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2024)
Acquiring annotations for whole slide images (WSIs)-based deep learning tasks, such as creating tissue segmentation masks or detecting mitotic figures, is a laborious process due to the extensive image size and the significant manual work involved in the annotation. This paper focuses on identifying and annotating specific image regions that optimize model training, given a limited annotation budget. While random sampling helps capture data variance by collecting annotation regions throughout the WSI, insufficient data curation may result in an inadequate representation of minority classes. Recent studies proposed diversity sampling to select a set of regions that maximally represent unique characteristics of the WSIs. This is done by pretraining on unlabeled data through self-supervised learning and then clustering all regions in the latent space. However, establishing the optimal number of clusters can be difficult and not all clusters are task-relevant. This paper presents prototype sampling, a new method for annotation region selection. It discovers regions exhibiting typical characteristics of each task-specific class. The process entails recognizing class prototypes from extensive histopathology image-caption databases and detecting unlabeled image regions that resemble these prototypes. Our results show that prototype sampling is more effective than random and diversity sampling in identifying annotation regions with valuable training information, resulting in improved model performance in semantic segmentation and mitotic figure detection tasks. Code is available at https://github.com/DeepMicroscopy/Prototype-sampling.
CRIS-Team Scopus Importer:2024-11-15
Keywords: annotation region selection; histopathology; image caption; low-data learning; Whole slide images

McLabel: A Local Thresholding Tool for Efficient Semi-automatic Labelling of Cells in Fluorescence Microscopy - Utz, Jonas; Schlereth, Maja; Qiu, Jingna; Thies, Mareike; Wagner, Fabian; Brahim, Oumaima B.; Gu, Mingxuan; Uderhardt, Stefan; Breininger, Katharina (Informatik aktuell, 2023)
In this work, we present a semi-automatic labelling tool for the annotation of complex cellular structures such as macrophages in fluorescence microscopy images. We present McLabel, a napari plugin that allows users to label structures of interest by simply scribbling outlines around the area of interest, using the triangle thresholding method with post-processing to identify the desired structure. Additionally, manual adaption of the threshold allows for quick and fine-grained local correction of the segmentation. The tool is evaluated in a user study with five experts, who annotated images both with and without the tool. The results show that variability in annotations between experts is reduced when the labelling tool is used and annotation time is reduced by a factor of five on average1.
CRIS-Team Scopus Importer:2023-09-29

Neural Networks for Distinguishing Rheumatoid Arthritis from Psoriatic Arthritis by Using Magnetic Resonance Imaging - Folle, Lukas; Bayat, Sara; Kleyer, Arnd; Fagni, Filippo; Kapsner, Lorenz; Schlereth, Maja; Meinderink, Timo; Breininger, Katharina; Tascilar, Koray; Krönke, Gerhard; Uder, Michael; Sticherling, Michael; et al. (Arthritis Rheumatol., 2022)