Abstract: Leveraging Image Captions for Selective Whole Slide Image Annotation - Qiu, Jingna; Aubreville, Marc; Wilm, Frauke; Öttl, Mathias; Utz, Jonas; Schlereth, Maja; Breininger, Katharina (Bildverarbeitung für die Medizin 2025, 2025)
Obtaining dense annotations for histopathological whole-slide images (WSI), such as segmentation masks or mitotic figure identification, is a labor intensive process due to the large image size and the extensive manual effort required for annotation. Identifying informative regions in WSIs for annotation while leaving other regions unlabeled can significantly reduce the annotation effort. These selected annotation regions should contain valuable training information that allows proper model training without significantly impacting performance compared to full annotatio

Abstract: Pan-tumor CAnine CuTaneous Cancer Histology (CATCH) Dataset - Wilm, Frauke; Fragoso, Marco; Marzahl, Christian; Qiu, Jingna; Puget, Chloé; Diehl, Laura; Bertram, Christof A.; Klopfleisch, Robert; Maier, Andreas; Breininger, Katharina; Aubreville, Marc (Informatik aktuell, 2023)
The identification of tumor regions on cutaneous tissue sections and the subsequent differentiation into individual tumor subtypes are routine tasks for veterinary pathologists. However, manual tumor delineation can be time-consuming and morphological similarities of tumor types can make the subtyping task difficult. Deep learning-based algorithms have been successfully applied for both tasks but require a large amount of annotated data to learn robust representations. Acquiring the annotations necessary to train these algorithms can be time-consuming and requires expert knowledge. Publicly available datasets reduce this annotation overhead and allow for a comparison of image analysis algorithms trained for the task at hand. We published a publicly available dataset of 350 whole slide images (WSIs) of seven different canine cutaneous tumors. These were annotated for the tumor subtypes and additionally six skin tissue classes. Overall, the database includes 12,424 polygon annotations for 13 classes, which exceeds most publicly available datasets in annotation extent and label diversity. In our dataset report, we validated the provided annotations through inter-rater experiments on a subset of the presented dataset, where three pathologists demonstrated high consistency in labeling tissue structures, especially for tumor annotations. Furthermore, we performed a technical validation by first training a deep neural network for tissue phenotyping. The model segmented the WSI into tumor and five skin tissue classes, where we achieved a class-averaged Jaccard coefficient of 0.7047 and a Jaccard coefficient of 0.9044 for tumor in particular. Afterward, we classified the segmented tumor regions into the tumor subtypes and achieved a slide-level accuracy of 0.9857. These baseline results provide a starting point for the development of more advanced algorithms. Previous works have shown that canine and human cutaneous tumors share various histologic patterns, which extends the added value of the presented dataset beyond veterinary pathology [1].
Created from Fastlane, Scopus look-up

Abstract: the MIDOG Challenge 2021 Mitosis Domain Generalization in Histopathology Images - Aubreville, Marc; Stathonikos, Nikolas; Bertram, Christof A.; Klopfleisch, Robert; ter Hoeve, Natalie; Ciompi, Francesco; Wilm, Frauke; Marzahl, Christian; Donovan, Taryn A.; Maier, Andreas; Veta, Mitko; Breininger, Katharina (Informatik aktuell, 2023)
The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor malignancy and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MItosis DOmain Generalization (MIDOG) 2021 challenge was the creation of scanner-agnostic MF detection algorithms. It was the first challenge to be held on the topic of histology domain generalization. The challenge used a training set of 200 cases, split across four scanning systems. The test set comprised an additional 100 cases split across four scanning systems, including two previously unseen scanners. We evaluated and compared the approaches that were submitted to the challenge and identified methodological factors contributing to better performance. The winning algorithm yielded an F1 score of 0.748 (CI95: 0.704-0.781). Additionally, we compared the performance of the algorithms to six pathologists. Irrespective of whether the original challenge ground or a newly generated object-level consensus of five experts was used as the basis for evaluation, we found that the best algorithms achieved higher performance levels than all experts. Further, the best algorithm reached the highest F1 values on either definition of the ground truth, reaching values of 0.742 and 0.748 on the five-expert consensus and the original challenge ground truth, respectively [1].
CRIS-Team Scopus Importer:2024-12-16

A comprehensive multi-domain dataset for mitotic figure detection - Aubreville, Marc; Wilm, Frauke; Stathonikos, Nikolas; Breininger, Katharina; Donovan, Taryn A.; Jabari, Samir; Veta, Mitko; Ganz, Jonathan; Ammeling, Jonas; van Diest, Paul J.; Klopfleisch, Robert; Bertram, Christof A. (Scientific Data, 2023)
The prognostic value of mitotic figures in tumor tissue is well-established for many tumor types and automating this task is of high research interest. However, especially deep learning-based methods face performance deterioration in the presence of domain shifts, which may arise from different tumor types, slide preparation and digitization devices. We introduce the MIDOG++ dataset, an extension of the MIDOG 2021 and 2022 challenge datasets. We provide region of interest images from 503 histological specimens of seven different tumor types with variable morphology with in total labels for 11,937 mitotic figures: breast carcinoma, lung carcinoma, lymphosarcoma, neuroendocrine tumor, cutaneous mast cell tumor, cutaneous melanoma, and (sub)cutaneous soft tissue sarcoma. The specimens were processed in several laboratories utilizing diverse scanners. We evaluated the extent of the domain shift by using state-of-the-art approaches, observing notable differences in single-domain training. In a leave-one-domain-out setting, generalizability improved considerably. This mitotic figure dataset is the first that incorporates a wide domain shift based on different tumor types, laboratories, whole slide image scanners, and species.
CRIS-Team Scopus Importer:2023-08-04

Adaptive Region Selection for Active Learning in Whole Slide Image Semantic Segmentation - Qiu, Jingna; Wilm, Frauke; Öttl, Mathias; Schlereth, Maja; Liu, Chang; Heimann, Tobias; Aubreville, Marc; Breininger, Katharina (The 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023), 2023)
The process of annotating histological gigapixel-sized whole slide images (WSIs) at the pixel level for the purpose of training a supervised segmentation model is time-consuming. Region-based active learning (AL) involves training the model on a limited number of annotated image regions instead of requesting annotations of the entire images. These annotation regions are iteratively selected, with the goal of optimizing model performance while minimizing the annotated area. The standard method for region selection evaluates the informativeness of all square regions of a specified size and then selects a specific quantity of the most informative regions. We find that the efficiency of this method highly depends on the choice of AL step size (i.e., the combination of region size and the number of selected regions per WSI), and a suboptimal AL step size can result in redundant annotation requests or inflated computation costs. This paper introduces a novel technique for selecting annotation regions adaptively, mitigating the reliance on this AL hyperparameter. Specifically, we dynamically determine each region by first identifying an informative area and then detecting its optimal bounding box, as opposed to selecting regions of a uniform predefined shape and size as in the standard method. We evaluate our method using the task of breast cancer metastases segmentation on the public CAMELYON16 dataset and show that it consistently achieves higher sampling efficiency than the standard method across various AL step sizes. With only 2.6% of tissue area annotated, we achieve full annotation performance and thereby substantially reduce the costs of annotating a WSI dataset. The source code is available at https://github.com/DeepMicroscopy/AdaptiveRegionSelection.
Keywords: Active learning; Region selection; Whole slide images.

Appearance-based Debiasing of Deep Learning Models in Medical Imaging - Wilm, Frauke; Reimann, Marcel Kim; Taubmann, Oliver; Mühlberg, Alexander; Breininger, Katharina (Bildverarbeitung für die Medizin 2024. BVM 2024, 2024)
Out-of-distribution data can substantially impede the performance of deep learning models. In medical imaging, domain shifts can, for instance, be caused by different image acquisition protocols. To address these domain shifts, domain adversarial training can be employed to constrain a model to domainagnostic features. This, however, requires prior knowledge about the domain variable, which might not always be accessible. Recent approaches make use of control regions to guide the training process and thereby alleviate the need for prior domain knowledge. In this work, we combine these approaches with traditional domain adversarial training to exploit the benefits of both methods.We test the proposed method on two medical datasets and demonstrate performance increases of up to 10 %, compared to a baseline trained without debiasing.
CRIS-Team Scopus Importer:2024-03-29

Author Correction: A comprehensive multi-domain dataset for mitotic figure detection (Scientific Data, (2023), 10, 1, (484), 10.1038/s41597-023-02327-4) - Aubreville, Marc; Wilm, Frauke; Stathonikos, Nikolas; Breininger, Katharina; Donovan, Taryn A.; Jabari, Samir; Veta, Mitko; Ganz, Jonathan; Ammeling, Jonas; van Diest, Paul J.; Klopfleisch, Robert; Bertram, Christof A. (Scientific Data, 2024)
A bug in the original code for calculating the average precision values, reported as mean and standard deviation in Tables 5 and 7, meant that these values were incorrect in the original work. They have been updated in the PDF and HTML versions of the article. Copies of the old and new tables are appended below to highlight the changes. The updated code is available at https://github.com/DeepMicroscopy/MIDOGpp alongside the previous version (Commits f6cd999 and 8b15a5a). Table 5 – old (Table presented.) breast carcinoma (human) lung carcinoma (canine) lymphosarcoma (canine) cutaneous mast cell tumor (canine) neuroendocrine tumor (human) soft tissue sarcoma (canine) melanoma (human) mean std mean std mean std mean std mean std mean std mean std breast carcinoma (human) 0.54 0.04 0.28 0.07 0.22 0.05 0.43 0.06 0.28 0.05 0.29 0.03 0.63 0.04 lung carcinoma (canine) 0.32 0.01 0.49 0.04 0.38 0.02 0.30 0.06 0.16 0.03 0.39 0.03 0.40 0.03 lymphosarcoma (canine) 0.24 0.03 0.26 0.04 0.48 0.04 0.27 0.05 0.04 0.03 0.17 0.03 0.18 0.06 cutaneous mast cell tumor (canine) 0.37 0.05 0.22 0.08 0.23 0.07 0.38 0.05 0.15 0.06 0.25 0.04 0.36 0.15 neuroendocrine tumor (human) 0.29 0.02 0.17 0.05 0.22 0.04 0.32 0.09 0.30 0.02 0.21 0.07 0.59 0.05 soft tissue sarcoma (canine) 0.41 0.08 0.40 0.07 0.39 0.05 0.37 0.10 0.24 0.04 0.45 0.04 0.48 0.08 melanoma (human) 0.42 0.05 0.26 0.08 0.14 0.05 0.42 0.07 0.31 0.02 0.30 0.04 0.67 0.03 all 0.58 0.02 0.49 0.04 0.51 0.05 0.41 0.04 0.28 0.04 0.42 0.03 0.66 0.02 Table 5 – new (Table presented.) breast carcinoma (human) lung carcinoma (canine) lymphosarcoma (canine) cutaneous mast cell tumor (canine) neuroendocrine tumor (human) soft tissue sarcoma (canine) melanoma (human) mean std mean std mean std mean std mean std mean std mean std breast carcinoma (human) 0.70 0.03 0.31 0.09 0.22 0.06 0.64 0.09 0.53 0.05 0.43 0.09 0.75 0.05 lung carcinoma (canine) 0.45 0.02 0.62 0.02 0.46 0.06 0.31 0.06 0.53 0.05 0.59 0.03 0.77 0.01 lymphosarcoma (canine) 0.44 0.05 0.50 0.05 0.80 0.01 0.67 0.03 0.55 0.01 0.48 0.05 0.59 0.05 cutaneous mast cell tumor (canine) 0.45 0.02 0.23 0.09 0.23 0.08 0.87 0.02 0.50 0.06 0.40 0.06 0.70 0.01 neuroendocrine tumor (human) 0.40 0.08 0.34 0.07 0.28 0.10 0.32 0.11 0.52 0.03 0.40 0.02 0.67 0.03 soft tissue sarcoma (canine) 0.55 0.05 0.48 0.05 0.41 0.05 0.54 0.11 0.38 0.09 0.69 0.04 0.60 0.09 melanoma (human) 0.46 0.09 0.30 0.10 0.14 0.05 0.46 0.11 0.55 0.03 0.52 0.02 0.80 0.01 all 0.74 0.02 0.65 0.06 0.69 0.04 0.82 0.01 0.62 0.03 0.69 0.03 0.82 0.03 Table 7 – old (Table presented.) breast carcinoma (human) lung carcinoma (canine) lymphosarcoma (canine) cutaneous mast cell tumor (canine) neuroendocrine tumor (human) soft tissue sarcoma (canine) melanoma (human) mean std mean std mean std mean std mean std mean std mean std w/o breast carcinoma (human) 0.50 0.02 0.52 0.04 0.48 0.02 0.38 0.03 0.28 0.03 0.42 0.05 0.62 0.03 w/o lung carcinoma (canine) 0.52 0.03 0.37 0.08 0.45 0.07 0.38 0.08 0.27 0.05 0.37 0.09 0.63 0.06 w/o lymphosarcoma (canine) 0.55 0.05 0.44 0.03 0.41 0.01 0.46 0.09 0.31 0.05 0.42 0.03 0.65 0.04 w/o cutaneous mast cell tumor (canine) 0.57 0.01 0.50 0.01 0.51 0.01 0.50 0.04 0.30 0.03 0.44 0.03 0.67 0.03 w/o neuroendocrine tumor (human) 0.58 0.03 0.50 0.02 0.50 0.03 0.42 0.02 0.33 0.01 0.44 0.03 0.66 0.01 w/o soft tissue sarcoma (canine) 0.56 0.02 0.46 0.03 0.47 0.02 0.37 0.03 0.26 0.05 0.40 0.03 0.63 0.03 w/o melanoma (human) 0.55 0.04 0.43 0.07 0.43 0.05 0.37 0.04 0.23 0.05 0.38 0.03 0.61 0.04 Table 7 - new (Table presented.) breast carcinoma (human) lung carcinoma (canine) lymphosarcoma (canine) cutaneous mast cell tumor (canine) neuroendocrine tumor (human) soft tissue sarcoma (canine) melanoma (human) mean std mean std mean std mean std mean std mean std mean std w/o breast carcinoma (human) 0.70 0.02 0.66 0.03 0.72 0.02 0.82 0.02 0.61 0.02 0.67 0.05 0.80 0.02 w/o lung carcinoma (canine) 0.71 0.05 0.65 0.05 0.67 0.08 0.81 0.05 0.61 0.04 0.70 0.02 0.79 0.04 w/o lymphosarcoma (canine) 0.71 0.04 0.64 0.03 0.71 0.03 0.81 0.04 0.60 0.02 0.68 0.02 0.80 0.02 w/o cutaneous mast cell tumor (canine) 0.72 0.02 0.64 0.03 0.65 0.12 0.79 0.06 0.62 0.01 0.69 0.01 0.82 0.02 w/o neuroendocrine tumor (human) 0.73 0.03 0.67 0.02 0.73 0.02 0.83 0.01 0.63 0.02 0.69 0.03 0.81 0.01 w/o soft tissue sarcoma (canine) 0.71 0.01 0.63 0.04 0.70 0.04 0.79 0.06 0.61 0.03 0.68 0.02 0.78 0.02 w/o melanoma (human) 0.71 0.04 0.59 0.08 0.60 0.16 0.77 0.05 0.60 0.03 0.67 0.04 0.79 0.03
CRIS-Team Scopus Importer:2024-07-12

Automated diagnosis of 7 canine skin tumors using machine learning on H&E-stained whole slide images - Fragoso-Garcia, Marco; Wilm, Frauke; Bertram, Christof A.; Merz, Sophie; Schmidt, Anja; Donovan, Taryn; Fuchs-Baumgartinger, Andrea; Bartel, Alexander; Marzahl, Christian; Diehl, Laura; Puget, Chloe; Maier, Andreas; et al. (Veterinary Pathology, 2023)
Microscopic evaluation of hematoxylin and eosin-stained slides is still the diagnostic gold standard for a variety of diseases, including neoplasms. Nevertheless, intra- and interrater variability are well documented among pathologists. So far, computer assistance via automated image analysis has shown potential to support pathologists in improving accuracy and reproducibility of quantitative tasks. In this proof of principle study, we describe a machine-learning-based algorithm for the automated diagnosis of 7 of the most common canine skin tumors: trichoblastoma, squamous cell carcinoma, peripheral nerve sheath tumor, melanoma, histiocytoma, mast cell tumor, and plasmacytoma. We selected, digitized, and annotated 350 hematoxylin and eosin-stained slides (50 per tumor type) to create a database divided into training, n = 245 whole-slide images (WSIs), validation (n = 35 WSIs), and test sets (n = 70 WSIs). Full annotations included the 7 tumor classes and 6 normal skin structures. The data set was used to train a convolutional neural network (CNN) for the automatic segmentation of tumor and nontumor classes. Subsequently, the detected tumor regions were classified patch-wise into 1 of the 7 tumor classes. A majority of patches-approach led to a tumor classification accuracy of the network on the slide-level of 95% (133/140 WSIs), with a patch-level precision of 85%. The same 140 WSIs were provided to 6 experienced pathologists for diagnosis, who achieved a similar slide-level accuracy of 98% (137/140 correct majority votes). Our results highlight the feasibility of artificial intelligence-based methods as a support tool in diagnostic oncologic pathology with future applications in other species and tumor types.
CRIS-Team Scopus Importer:2023-08-11
Keywords: computational pathology; computer-aided diagnosis; digital pathology; dog; machine learning; skin; veterinary oncology

Breast cancer survival prediction using an automated mitosis detection pipeline - Stathonikos, Nikolas; Aubreville, Marc; de Vries, Sjoerd; Wilm, Frauke; Bertram, Christof A.; Veta, Mitko; van Diest, Paul J. (Journal of Pathology: Clinical Research, 2024)
Mitotic count (MC) is the most common measure to assess tumor proliferation in breast cancer patients and is highly predictive of patient outcomes. It is, however, subject to inter- and intraobserver variation and reproducibility challenges that may hamper its clinical utility. In past studies, artificial intelligence (AI)-supported MC has been shown to correlate well with traditional MC on glass slides. Considering the potential of AI to improve reproducibility of MC between pathologists, we undertook the next validation step by evaluating the prognostic value of a fully automatic method to detect and count mitoses on whole slide images using a deep learning model. The model was developed in the context of the Mitosis Domain Generalization Challenge 2021 (MIDOG21) grand challenge and was expanded by a novel automatic area selector method to find the optimal mitotic hotspot and calculate the MC per 2 mm2. We employed this method on a breast cancer cohort with long-term follow-up from the University Medical Centre Utrecht (N = 912) and compared predictive values for overall survival of AI-based MC and light-microscopic MC, previously assessed during routine diagnostics. The MIDOG21 model was prognostically comparable to the original MC from the pathology report in uni- and multivariate survival analysis. In conclusion, a fully automated MC AI algorithm was validated in a large cohort of breast cancer with regard to retained prognostic value compared with traditional light-microscopic MC.
CRIS-Team Scopus Importer:2024-11-08
Keywords: artificial intelligence; deep learning; histopathology; machine learning; mitosis; outcome; prognosis

Cell Detection for Asthma on Partially Annotated Whole Slide Images: Learning to be EXACT - Marzahl, Christian; Bertram, Christof A.; Wilm, Frauke; Voigt, Jörn; Barton, Ann K.; Klopfleisch, Robert; Breininger, Katharina; Maier, Andreas; Aubreville, Marc (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

Computer Assistance: A New Gold Standard for the Mitotic Count? - Bertram, Christof A.; Aubreville, Marc; Donovan, Taryn A.; Bartel, Alexander; Wilm, Frauke; Marzahl, Christian; Assenmacher, Charles-Antoine; Becker, Kathrin; Bennett, Mark; Corner, Sarah; Cossic, Brieuc; Denk, Daniela; et al. (Journal of Comparative Pathology, 2022)

Computer-assisted mitotic count using a deep learning–based algorithm improves interobserver reproducibility and accuracy - Bertram, Christof A.; Aubreville, Marc; Donovan, Taryn A.; Bartel, Alexander; Wilm, Frauke; Marzahl, Christian; Assenmacher, Charles-Antoine; Becker, Kathrin; Bennett, Mark; Corner, Sarah; Cossic, Brieuc; Denk, Daniela; et al. (Veterinary Pathology, 2022)
The mitotic count (MC) is an important histological parameter for prognostication of malignant neoplasms. However, it has inter- and intraobserver discrepancies due to difficulties in selecting the region of interest (MC-ROI) and in identifying or classifying mitotic figures (MFs). Recent progress in the field of artificial intelligence has allowed the development of high-performance algorithms that may improve standardization of the MC. As algorithmic predictions are not flawless, computer-assisted review by pathologists may ensure reliability. In the present study, we compared partial (MC-ROI preselection) and full (additional visualization of MF candidates and display of algorithmic confidence values) computer-assisted MC analysis to the routine (unaided) MC analysis by 23 pathologists for whole-slide images of 50 canine cutaneous mast cell tumors (ccMCTs). Algorithmic predictions aimed to assist pathologists in detecting mitotic hotspot locations, reducing omission of MFs, and improving classification against imposters. The interobserver consistency for the MC significantly increased with computer assistance (interobserver correlation coefficient, ICC = 0.92) compared to the unaided approach (ICC = 0.70). Classification into prognostic stratifications had a higher accuracy with computer assistance. The algorithmically preselected hotspot MC-ROIs had a consistently higher MCs than the manually selected MC-ROIs. Compared to a ground truth (developed with immunohistochemistry for phosphohistone H3), pathologist performance in detecting individual MF was augmented when using computer assistance (F1-score of 0.68 increased to 0.79) with a reduction in false negatives by 38%. The results of this study demonstrate that computer assistance may lead to more reproducible and accurate MCs in ccMCTs.
CRIS-Team Scopus Importer:2022-03-25
Keywords: artificial intelligence; automated image analysis; canine cutaneous mast cell tumors; computer assistance; deep learning; digital pathology; mitotic count; mitotic figures

Dataset on Bi- and Multi-nucleated Tumor Cells in Canine Cutaneous Mast Cell Tumors - Bertram, Christof A.; Donovan, Taryn A.; Tecilla, Marco; Bartenschlager, Florian; Fragoso, Marco; Wilm, Frauke; Marzahl, Christian; Breininger, Katharina; Maier, Andreas; Klopfleisch, Robert; Aubreville, Marc (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

Domain Adversarial RetinaNet as a Reference Algorithm for the MItosis DOmain Generalization Challenge - Wilm, Frauke; Marzahl, Christian; Breininger, Katharina; Aubreville, Marc (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2022)
Assessing the mitotic count has a known high degree of intra- and inter-rater variability. Computer-aided systems have proven to decrease this variability and reduce labeling time. These systems, however, are generally highly dependent on their training domain and show poor applicability to unseen domains. In histopathology, these domain shifts can result from various sources, including different slide scanning systems used to digitize histologic samples. The MItosis DOmain Generalization challenge focused on this specific domain shift for the task of mitotic figure detection. This work presents a mitotic figure detection algorithm developed as a baseline for the challenge, based on domain adversarial training. On the challenge’s test set, the algorithm scored an F 1 score of 0.7183. The corresponding network weights and code for implementing the network are made publicly available.
CRIS-Team Scopus Importer:2022-03-25
Keywords: Domain Shift; Histopathology; MIDOG; Mitotic Count; Object Detection

Domain generalization across tumor types, laboratories, and species — Insights from the 2022 edition of the Mitosis Domain Generalization Challenge - Aubreville, Marc; Stathonikos, Nikolas; Donovan, Taryn A.; Klopfleisch, Robert; Ammeling, Jonas; Ganz, Jonathan; Wilm, Frauke; Veta, Mitko; Jabari, Samir; Eckstein, Markus; Annuscheit, Jonas; Krumnow, Christian; et al. (Medical Image Analysis, 2024)
Recognition of mitotic figures in histologic tumor specimens is highly relevant to patient outcome assessment. This task is challenging for algorithms and human experts alike, with deterioration of algorithmic performance under shifts in image representations. Considerable covariate shifts occur when assessment is performed on different tumor types, images are acquired using different digitization devices, or specimens are produced in different laboratories. This observation motivated the inception of the 2022 challenge on MItosis Domain Generalization (MIDOG 2022). The challenge provided annotated histologic tumor images from six different domains and evaluated the algorithmic approaches for mitotic figure detection provided by nine challenge participants on ten independent domains. Ground truth for mitotic figure detection was established in two ways: a three-expert majority vote and an independent, immunohistochemistry-assisted set of labels. This work represents an overview of the challenge tasks, the algorithmic strategies employed by the participants, and potential factors contributing to their success. With an F1 score of 0.764 for the top-performing team, we summarize that domain generalization across various tumor domains is possible with today's deep learning-based recognition pipelines. However, we also found that domain characteristics not present in the training set (feline as new species, spindle cell shape as new morphology and a new scanner) led to small but significant decreases in performance. When assessed against the immunohistochemistry-assisted reference standard, all methods resulted in reduced recall scores, with only minor changes in the order of participants in the ranking.
CRIS-Team Scopus Importer:2024-04-05
Keywords: Challenge; Deep Learning; Domain generalization; Histopathology; Mitosis

Fast whole-slide cartography in colon cancer histology using superpixels and CNN classification - Wilm, Frauke; Benz, Michaela; Bruns, Volker; Baghdadlian, Serop; Dexl, Jakob; Hartmann, David; Kuritcyn, Petr; Weidenfeller, Martin; Wittenberg, Thomas; Merkel, Susanne; Hartmann, Arndt; Eckstein, Markus; et al. (Journal of Medical Imaging, 2022)
Purpose: Automatic outlining of different tissue types in digitized histological specimen provides a basis for follow-up analyses and can potentially guide subsequent medical decisions. The immense size of whole-slide-images (WSIs), however, poses a challenge in terms of computation time. In this regard, the analysis of nonoverlapping patches outperforms pixelwise segmentation approaches but still leaves room for optimization. Furthermore, the division into patches, regardless of the biological structures they contain, is a drawback due to the loss of local dependencies.
CRIS-Team WoS Importer:2022-05-13

Fully automatic HER2 tissue segmentation for interpretable HER2 scoring - Öttl, Mathias; Steenpaß, Jana; Wilm, Frauke; Qiu, Jingna; Rübner, Matthias; Lang-Schwarz, Corinna; Taverna, Cecilia; Tava, Francesca; Hartmann, Arndt; Hübner, Hanna; Beckmann, Matthias; Fasching, Peter; et al. (Journal of Pathology Informatics, 2025)
Breast cancer is the most common cancer in women, with HER2 (human epidermal growth factor receptor 2) overexpression playing a critical role in regulating cell growth and division. HER2 status, assessed according to established scoring guidelines, offers important information for treatment selection. However, the complexity of the task leads to variability in human rater assessments. In this work, we propose a fully automated, interpretable HER2 scoring pipeline based on pixel-level semantic segmentations, designed to align with clinical guidelines. Using polygon annotations, our method balances annotation effort with the ability to capture fine-grained details and larger structures, such as non-invasive tumor tissue. To enhance HER2 segmentation, we propose the use of a Wasserstein Dice loss to model class relationships, ensuring robust segmentation and HER2 scoring performance. Additionally, based on observations of pathologists' behavior in clinical practice, we propose a calibration step to the scoring rules, which positively impacts the accuracy and consistency of automated HER2 scoring. Our approach achieves an F1 score of 0.832 on HER2 scoring, demonstrating its effectiveness. This work establishes a potent segmentation pipeline that can be further leveraged to analyze HER2 expression in breast cancer tissue.
CRIS-Team Scopus Importer:2025-04-11
Keywords: Deep learning; HER2; HER2 scoring; Histopathology; Semantic segmentation

Improved HER2 Tumor Segmentation with Subtype Balancing Using Deep Generative Networks - Öttl, Mathias; Steenpaß, Jana; Rübner, Matthias; Geppert, Carol-Immanuel; Qiu, Jingna; Wilm, Frauke; Hartmann, Arndt; Beckmann, Matthias; Fasching, Peter; Maier, Andreas; Erber, Ramona; Breininger, Katharina (Proceedings - International Symposium on Biomedical Imaging, 2023)
Tumor segmentation in histopathology images is often complicated by its composition of different histological subtypes and class imbalance. Oversampling subtypes with low prevalence features is not a satisfactory solution since it eventually leads to overfitting. We propose to create synthetic images with semantically-conditioned deep generative networks and to combine subtype-balanced synthetic images with the original dataset to achieve better segmentation performance. We show the suitability of Generative Adversarial Networks (GANs) and especially diffusion models to create realistic images based on subtype-conditioning for the use case of HER2-stained histopathology. Additionally, we show the capability of diffusion models to conditionally inpaint HER2 tumor areas with modified subtypes. Combining the original dataset with the same amount of diffusion-generated images increased the tumor Dice score from 0.833 to 0.854 and almost halved the variance between the HER2 subtype recalls. These results create the basis for more reliable automatic HER2 analysis with lower performance variance between individual HER2 subtypes.
CRIS-Team Scopus Importer:2023-10-06
Keywords: Diffusion Models; Generative Models; HER2; Histopathology; Segmentation; Subtypes

Influence of Inter-Annotator Variability on Automatic Mitotic Figure Assessment - Wilm, Frauke; Bertram, Christof A.; Marzahl, Christian; Bartel, Alexander; Donovan, Taryn A.; Assenmacher, Charles Antoine; Becker, Kathrin; Bennett, Mark; Corner, Sarah; Cossic, Brieuc; Denk, Daniela; Dettwiler, Martina; et al. (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

Inter-species cell detection - datasets on pulmonary hemosiderophages in equine, human and feline specimens - Marzahl, Christian; Hill, Jenny; Stayt, Jason; Bienzle, Dorothee; Welker, Lutz; Wilm, Frauke; Voigt, Joern; Aubreville, Marc; Maier, Andreas; Klopfleisch, Robert; Breininger, Katharina; Bertram, Christof A. (Scientific Data, 2022)
Pulmonary hemorrhage (P-Hem) occurs among multiple species and can have various causes. Cytology of bronchoalveolar lavage fluid (BALF) using a 5-tier scoring system of alveolar macrophages based on their hemosiderin content is considered the most sensitive diagnostic method. We introduce a novel, fully annotated multi-species P-Hem dataset, which consists of 74 cytology whole slide images (WSIs) with equine, feline and human samples. To create this high-quality and high-quantity dataset, we developed an annotation pipeline combining human expertise with deep learning and data visualisation techniques. We applied a deep learning-based object detection approach trained on 17 expertly annotated equine WSIs, to the remaining 39 equine, 12 human and 7 feline WSIs. The resulting annotations were semi-automatically screened for errors on multiple types of specialised annotation maps and finally reviewed by a trained pathologist. Our dataset contains a total of 297,383 hemosiderophages classified into five grades. It is one of the largest publicly available WSIs datasets with respect to the number of annotations, the scanned area and the number of species covered.
CRIS-Team WoS Importer:2022-06-17

Iterative Cross-Scanner Registration for Whole Slide Images - Theelke, Luisa; Wilm, Frauke; Marzahl, Christian; Bertram, Christof A.; Klopfleisch, Robert; Maier, Andreas; Aubreville, Marc; Breininger, Katharina (2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021), 2021)
The successful registration of digitized microscopic images is required for many applications in digital pathology. In particular, the registration of specimens scanned by different slide scanning systems may be beneficial to transfer expert annotations from one image domain to another and thereby reduce labeling effort. We present an iterative approach to register microscopic specimens digitized with multiple scanning systems, aiming to compute an optimal global transformation for the images at highest resolution. For this purpose, an initial registration based on a down-scaled version of the images is followed by a patch-based iterative update scheme. We make use of the hierarchical structure of digitized whole slide images to gradually approximate the optimal transformation. By using kernel density estimation to weight local transformation estimates, the influence of registration errors can be further mitigated. We validate our method on five histologic and five cytologic samples, each scanned with four different scanning systems. Furthermore, we perform first experiments on samples stained with different stain combinations. Our experiments demonstrate the potential of the proposed method for a variety of datasets and application fields.<br
CRIS-Team WoS Importer:2022-02-25

Leveraging Image Captions for Selective Whole Slide Image Annotation - Qiu, Jingna; Aubreville, Marc; Wilm, Frauke; Öttl, Mathias; Utz, Jonas; Schlereth, Maja; Breininger, Katharina (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2024)
Acquiring annotations for whole slide images (WSIs)-based deep learning tasks, such as creating tissue segmentation masks or detecting mitotic figures, is a laborious process due to the extensive image size and the significant manual work involved in the annotation. This paper focuses on identifying and annotating specific image regions that optimize model training, given a limited annotation budget. While random sampling helps capture data variance by collecting annotation regions throughout the WSI, insufficient data curation may result in an inadequate representation of minority classes. Recent studies proposed diversity sampling to select a set of regions that maximally represent unique characteristics of the WSIs. This is done by pretraining on unlabeled data through self-supervised learning and then clustering all regions in the latent space. However, establishing the optimal number of clusters can be difficult and not all clusters are task-relevant. This paper presents prototype sampling, a new method for annotation region selection. It discovers regions exhibiting typical characteristics of each task-specific class. The process entails recognizing class prototypes from extensive histopathology image-caption databases and detecting unlabeled image regions that resemble these prototypes. Our results show that prototype sampling is more effective than random and diversity sampling in identifying annotation regions with valuable training information, resulting in improved model performance in semantic segmentation and mitotic figure detection tasks. Code is available at https://github.com/DeepMicroscopy/Prototype-sampling.
CRIS-Team Scopus Importer:2024-11-15
Keywords: annotation region selection; histopathology; image caption; low-data learning; Whole slide images

"MigraineMonitor" – Towards a System for the Prediction of Migraine Attacks using Electrostimulation - Stefke, Andrea; Wilm, Frauke; Richer, Robert; Gradl, Stefan; Eskofier, Björn; Forster, Clemens; Namer, Barbara (Current Directions in Biomedical Engineering, 2018)
Migraine attacks can be accompanied by many different symptoms, some of them appearing within 24 hours before the onset of the headache. In previous work, reduced habituation to an electrical pain stimulus at the head was observed in the pre-ictal phase within 24 hours before the headache attack. Based on these results, this work presents an application to track influence factors on migraine attacks and an Arduino-based control unit which replaces the traditional approach of manual electrical stimulation. The usability of both components of the project was evaluated in separate user studies. Results of the usability study show a good acceptance of the systems with a mean SUS score of 92.4. Additionally, they indicate that the developed control unit may substitute the current manual electrical stimulation. Overall, the designed system allows standardized repeatable measurements and is a first step towards the home-use of a device for establishing a new method for migraine prediction.
Keywords: migraine prediction; migraine tracking; premonitory symptoms; electrostimulation; smartphone application

Mind the Gap: Scanner-Induced Domain Shifts Pose Challenges for Representation Learning in Histopathology - Wilm, Frauke; Fragoso, Marco; Bertram, Christof A.; Stathonikos, Nikolas; Ottl, Mathias; Qiu, Jingna; Klopfleisch, Robert; Maier, Andreas; Aubreville, Marc; Breininger, Katharina (Proceedings - International Symposium on Biomedical Imaging, 2023)
Computer-aided systems in histopathology are often challenged by various sources of domain shift that impact the performance of these algorithms considerably. We investigated the potential of using self-supervised pre-training to overcome scanner-induced domain shifts for the downstream task of tumor segmentation. For this, we present the Barlow Triplets to learn scanner-invariant representations from a multi-scanner dataset with local image correspondences. We show that self-supervised pre-training successfully aligned different scanner representations, which, interestingly only results in a limited benefit for our downstream task. We thereby provide insights into the influence of scanner characteristics for downstream applications and contribute to a better understanding of why established self-supervised methods have not yet shown the same success on histopathology data as they have for natural images.
Created from Fastlane, Scopus look-up
Keywords: Barlow Twins; Domain Shift; Histopathology; Representation Learning

Mitosis domain generalization in histopathology images — The MIDOG challenge - Aubreville, Marc; Stathonikos, Nikolas; Bertram, Christof A.; Klopfleisch, Robert; ter Hoeve, Natalie; Ciompi, Francesco; Wilm, Frauke; Marzahl, Christian; Donovan, Taryn A.; Maier, Andreas; Breen, Jack; Ravikumar, Nishant; et al. (Medical Image Analysis, 2023)
The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate when applied in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MIDOG 2021 challenge was the creation of scanner-agnostic MF detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were provided. In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance. The winning algorithm yielded an F1 score of 0.748 (CI95: 0.704-0.781), exceeding the performance of six experts on the same task.
CRIS-Team Scopus Importer:2022-12-16
Keywords: Challenge; Deep Learning; Domain generalization; Histopathology; Mitosis

Multi-level cancer profiling through joint cell-graph representations - Rivera Monroy, Luis Carlos; Rist, Leonhard; Wilm, Frauke; Ostalecki, Christian; Baur, Andreas; Vera González, Julio; Breininger, Katharina; Maier, Andreas (Smart Health, 2024)
Computer-aided analysis of digitized pathology samples has significantly advanced with the rapid progression of machine and Deep Learning (DL) methods. However, most existing approaches primarily focus on features extracted from patches due to the large image sizes. This focus limits the ability of Convolutional Neural Networks (CNNs) to capture global information from the samples, resulting in an incomplete phenotypical and topological representation and thereby restricting the diagnostic capabilities of these methods. The recent emergence of Graph Neural Networks (GNNs) offers new opportunities to overcome these limitations through graph-driven representations of pathological samples. This work introduces a graph-based framework that encompasses diverse cancer types and integrates different imaging modalities. In this framework, histopathology samples are represented as graphs, and a pipeline facilitating cell-wise and disease classification is developed. The results support this motivation: for cell-wise classification, we achieved an average accuracy of , and for disease-wise classification, an average accuracy of 83%, outperforming reference models such as XGBoost and standard CNNs. This approach not only provides flexibility in combining various diseases but also extends to integrating different staining technique
Keywords: Fluorescence microscopy imageGraph deep learningMultiplex imagingDigital pathology

Multi-scanner Canine Cutaneous Squamous Cell Carcinoma Histopathology Dataset - Wilm, Frauke; Fragoso, Marco; Bertram, Christof A.; Stathonikos, Nikolas; Öttl, Mathias; Qiu, Jingna; Klopfleisch, Robert; Maier, Andreas; Breininger, Katharina; Aubreville, Marc (Informatik aktuell, 2023)
In histopathology, scanner-induced domain shifts are known to impede the performance of trained neural networks when tested on unseen data. Multidomain pre-training or dedicated domain-generalization techniques can help to develop domain-agnostic algorithms. For this, multi-scanner datasets with a high variety of slide scanning systems are highly desirable. We present a publicly available multi-scanner dataset of canine cutaneous squamous cell carcinoma histopathology images, composed of 44 samples digitized with five slide scanners. This dataset provides local correspondences between images and thereby isolates the scanner-induced domain shift from other inherent, e.g. morphology-induced domain shifts. To highlight scanner differences, we present a detailed evaluation of color distributions, sharpness, and contrast of the individual scanner subsets. Additionally, to quantify the inherent scanner-induced domain shift, we train a tumor segmentation network on each scanner subset and evaluate the performance both in - and cross-domain. We achieve a class-averaged in-domain intersection over union coefficient of up to 0.86 and observe a cross-domain performance decrease of up to 0.38, which confirms the inherent domain shift of the presented dataset and its negative impact on the performance of deep neural networks.
Created from Fastlane, Scopus look-up

Pan-tumor CAnine cuTaneous Cancer Histology (CATCH) dataset - Wilm, Frauke; Fragoso, Marco; Marzahl, Christian; Qiu, Jingna; Puget, Chloe; Diehl, Laura; Bertram, Christof A.; Klopfleisch, Robert; Maier, Andreas; Breininger, Katharina; Aubreville, Marc (Scientific Data, 2022)
Due to morphological similarities, the differentiation of histologic sections of cutaneous tumors into individual subtypes can be challenging. Recently, deep learning-based approaches have proven their potential for supporting pathologists in this regard. However, many of these supervised algorithms require a large amount of annotated data for robust development. We present a publicly available dataset of 350 whole slide images of seven different canine cutaneous tumors complemented by 12,424 polygon annotations for 13 histologic classes, including seven cutaneous tumor subtypes. In inter-rater experiments, we show a high consistency of the provided labels, especially for tumor annotations. We further validate the dataset by training a deep neural network for the task of tissue segmentation and tumor subtype classification. We achieve a class-averaged Jaccard coefficient of 0.7047, and 0.9044 for tumor in particular. For classification, we achieve a slide-level accuracy of 0.9857. Since canine cutaneous tumors possess various histologic homologies to human tumors the added value of this dataset is not limited to veterinary pathology but extends to more general fields of application.
CRIS-Team Scopus Importer:2022-10-14

Pan-tumor T-lymphocyte detection using deep neural networks: Recommendations for transfer learning in immunohistochemistry - Wilm, Frauke; Ihling, Christian; Méhes, Gábor; Terracciano, Luigi; Puget, Chloé; Klopfleisch, Robert; Schüffler, Peter; Aubreville, Marc; Maier, Andreas; Mrowiec, Thomas; Breininger, Katharina (Journal of Pathology Informatics, 2023)
The success of immuno-oncology treatments promises long-term cancer remission for an increasing number of patients. The response to checkpoint inhibitor drugs has shown a correlation with the presence of immune cells in the tumor and tumor microenvironment. An in-depth understanding of the spatial localization of immune cells is therefore critical for understanding the tumor's immune landscape and predicting drug response. Computer-aided systems are well suited for efficiently quantifying immune cells in their spatial context. Conventional image analysis approaches are often based on color features and therefore require a high level of manual interaction. More robust image analysis methods based on deep learning are expected to decrease this reliance on human interaction and improve the reproducibility of immune cell scoring. However, these methods require sufficient training data and previous work has reported low robustness of these algorithms when they are tested on out-of-distribution data from different pathology labs or samples from different organs. In this work, we used a new image analysis pipeline to explicitly evaluate the robustness of marker-labeled lymphocyte quantification algorithms depending on the number of training samples before and after being transferred to a new tumor indication. For these experiments, we adapted the RetinaNet architecture for the task of T-lymphocyte detection and employed transfer learning to bridge the domain gap between tumor indications and reduce the annotation costs for unseen domains. On our test set, we achieved human-level performance for almost all tumor indications with an average precision of 0.74 in-domain and 0.72–0.74 cross-domain. From our results, we derive recommendations for model development regarding annotation extent, training sample selection, and label extraction for the development of robust algorithms for immune cell scoring. By extending the task of marker-labeled lymphocyte quantification to a multi-class detection task, the pre-requisite for subsequent analyses, e.g., distinguishing lymphocytes in the tumor stroma from tumor-infiltrating lymphocytes, is met.
CRIS-Team Scopus Importer:2023-07-21
Keywords: Deep learning; Domain adaptation; Immuno-oncology; Immunohistochemistry; Transfer learning; Tumor-infiltrating lymphocytes

Reference Algorithms for the Mitosis Domain Generalization (MIDOG) 2022 Challenge - Ammeling, Jonas; Wilm, Frauke; Ganz, Jonathan; Breininger, Katharina; Aubreville, Marc (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2023)
Robust mitosis detection on images from different tumor types, pathology labs, and species is a challenging task that was addressed in the MICCAI Mitosis Domain Generalization (MIDOG) 2022 challenge. In this work, we describe three reference algorithms that were provided as a baseline for the challenge: A Mask-RCNN-based instance segmentation model trained on the MIDOG 2022 dataset, and two different versions of the domain-adversarial RetinaNet which already served as the baseline for MIDOG 2021 challenge, one trained on the MIDOG 2022 dataset and the other trained only on human breast carcinoma from MIDOG 2021. The domain-adversarial RetinaNet trained on the MIDOG 2022 dataset had the highest F 1 score of 0.7135 on the final test set. When trained on breast carcinoma only, the same network had a much lower F 1 score of 0.4719, indicating a significant domain shift between mitotic figure and tissue representation in different tumor types.
CRIS-Team Scopus Importer:2023-10-20

Robust Quad-Tree based Registration on Whole Slide Images - Marzahl, Christian; Wilm, Frauke; Dressler, Franz F.; Tharun, Lars; Perner, Sven; Bertram, Christof A.; Kröger, Christine; Voigt, Jörn; Klopfleisch, Robert; Maier, Andreas; Aubreville, Marc; Breininger, Katharina (MICCAI Computational Pathlogy (COMPAY) Workshop, 2021)
The registration of whole slide images (WSIs) provides the basis for many subsequent processing steps in digital pathology. For instance, the registration of immunohistochemistry (IHC) and hematoxylin & eosin (H&E)-stained WSIs is usually the first step in guiding IHC diagnostic procedures. Still, many registration methods operate poorly on WSIs. Reasons for this include the WSI size, fluctuating image quality or elastic tissue deformations. Multiple prior methods are further specialised towards a specific image modality, such as histology or cytology, or rely on a specific preparation protocol. To minimise these effects, we developed a robust WSI registration, which differs from previous methods by the following new aspect: We introduce a multi-scale approach based on a quad-tree (QT), with several termination criteria that makes the algorithm particularly insensitive to tissue artefacts and that further allows to estimate a piece-wise affine transformation. We validated our method on five scanner systems and 60 WSIs with different stainings. Our results outperformed any publicly available WSI registration method. The QT code, WSI landmarks and tools used to create the validation dataset are made publicly available.<br

Style-Extracting Diffusion Models for Semi-supervised Histopathology Segmentation - Öttl, Mathias; Wilm, Frauke; Steenpaß, Jana; Qiu, Jingna; Rübner, Matthias; Hartmann, Arndt; Beckmann, Matthias; Fasching, Peter; Maier, Andreas; Erber, Ramona; Kainz, Bernhard; Breininger, Katharina (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2025)
Deep learning-based image generation has seen significant advancements with diffusion models, notably improving the quality of generated images. Despite these developments, generating images with unseen characteristics beneficial for downstream tasks has received limited attention. To bridge this gap, we propose Style-Extracting Diffusion Models, featuring two conditioning mechanisms. Specifically, we utilize 1) a style conditioning mechanism which allows to inject style information of previously unseen images during image generation and 2) a content conditioning which can be targeted to a downstream task, e.g., layout for segmentation. We introduce a trainable style encoder to extract style information from images, and an aggregation block that merges style information from multiple style inputs. This architecture enables the generation of images with unseen styles in a zero-shot manner, by leveraging styles from unseen images, resulting in more diverse generations. In this work, we use the image layout as target condition and first show the capability of our method on a natural image dataset as a proof-of-concept. We further demonstrate its versatility in histopathology, where we combine prior knowledge about tissue composition and unannotated data to create diverse synthetic images with known layouts. This allows us to generate additional synthetic data to train a segmentation network in a semi-supervised fashion. We verify the added value of the generated images by showing improved segmentation results and lower performance variability between patients when synthetic images are included during segmentation training. The code of the method is publicly available at https://github.com/OettlM/STEDM.
Created from Fastlane, Scopus look-up