Abstract: Are Fast Labeling Methods Reliable?: A Case Study of Computer-aided Expert Annotations on Microscopy Slides - Marzahl, Christian; Bertram, Christof A.; Aubreville, Marc; Petrick, Anne; Weiler, Kristina; Gläsel, Agnes C.; Fragoso, Marco; Merz, Sophie; Bartenschlager, Florian; Hoppe, Judith; Langenhagen, Alina; Jasensky, Anne Katherine; et al. (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

Abstract: Deep Learning Algorithms Out-perform Veterinary Pathologists in Detecting the Mitotically Most Active Tumor Region - Aubreville, Marc; Bertram, Christof A.; Marzahl, Christian; Gurtner, Corinne; Dettwiler, Martina; Schmidt, Anja; Bartenschlager, Florian; Merz, Sophie; Fragoso, Marco; Kershaw, Olivia; Klopeisch, Robert; Maier, Andreas (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-09

Abstract: Deep Learning-based Quantification of Pulmonary Hemosiderophages in Cytology Slides - Marzahl, Christian; Aubreville, Marc; Bertram, Christof A.; Stayt, Jason; Jasensky, Anne Katherine; Bartenschlager, Florian; Fragoso, Marco; Barton, Ann K.; Elsemann, Svenja; Jabari, Samir; Krauth, Jens; Madhu, Prathmesh; et al. (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

Abstract: Pan-tumor CAnine CuTaneous Cancer Histology (CATCH) Dataset - Wilm, Frauke; Fragoso, Marco; Marzahl, Christian; Qiu, Jingna; Puget, Chloé; Diehl, Laura; Bertram, Christof A.; Klopfleisch, Robert; Maier, Andreas; Breininger, Katharina; Aubreville, Marc (Informatik aktuell, 2023)
The identification of tumor regions on cutaneous tissue sections and the subsequent differentiation into individual tumor subtypes are routine tasks for veterinary pathologists. However, manual tumor delineation can be time-consuming and morphological similarities of tumor types can make the subtyping task difficult. Deep learning-based algorithms have been successfully applied for both tasks but require a large amount of annotated data to learn robust representations. Acquiring the annotations necessary to train these algorithms can be time-consuming and requires expert knowledge. Publicly available datasets reduce this annotation overhead and allow for a comparison of image analysis algorithms trained for the task at hand. We published a publicly available dataset of 350 whole slide images (WSIs) of seven different canine cutaneous tumors. These were annotated for the tumor subtypes and additionally six skin tissue classes. Overall, the database includes 12,424 polygon annotations for 13 classes, which exceeds most publicly available datasets in annotation extent and label diversity. In our dataset report, we validated the provided annotations through inter-rater experiments on a subset of the presented dataset, where three pathologists demonstrated high consistency in labeling tissue structures, especially for tumor annotations. Furthermore, we performed a technical validation by first training a deep neural network for tissue phenotyping. The model segmented the WSI into tumor and five skin tissue classes, where we achieved a class-averaged Jaccard coefficient of 0.7047 and a Jaccard coefficient of 0.9044 for tumor in particular. Afterward, we classified the segmented tumor regions into the tumor subtypes and achieved a slide-level accuracy of 0.9857. These baseline results provide a starting point for the development of more advanced algorithms. Previous works have shown that canine and human cutaneous tumors share various histologic patterns, which extends the added value of the presented dataset beyond veterinary pathology [1].
Created from Fastlane, Scopus look-up

Abstract: the MIDOG Challenge 2021 Mitosis Domain Generalization in Histopathology Images - Aubreville, Marc; Stathonikos, Nikolas; Bertram, Christof A.; Klopfleisch, Robert; ter Hoeve, Natalie; Ciompi, Francesco; Wilm, Frauke; Marzahl, Christian; Donovan, Taryn A.; Maier, Andreas; Veta, Mitko; Breininger, Katharina (Informatik aktuell, 2023)
The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor malignancy and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MItosis DOmain Generalization (MIDOG) 2021 challenge was the creation of scanner-agnostic MF detection algorithms. It was the first challenge to be held on the topic of histology domain generalization. The challenge used a training set of 200 cases, split across four scanning systems. The test set comprised an additional 100 cases split across four scanning systems, including two previously unseen scanners. We evaluated and compared the approaches that were submitted to the challenge and identified methodological factors contributing to better performance. The winning algorithm yielded an F1 score of 0.748 (CI95: 0.704-0.781). Additionally, we compared the performance of the algorithms to six pathologists. Irrespective of whether the original challenge ground or a newly generated object-level consensus of five experts was used as the basis for evaluation, we found that the best algorithms achieved higher performance levels than all experts. Further, the best algorithm reached the highest F1 values on either definition of the ground truth, reaching values of 0.742 and 0.748 on the five-expert consensus and the original challenge ground truth, respectively [1].
CRIS-Team Scopus Importer:2024-12-16

A completely annotated whole slide image dataset of canine breast cancer to aid human breast cancer research - Aubreville, Marc; Bertram, Christof A.; Donovan, Taryn A.; Marzahl, Christian; Maier, Andreas; Klopfleisch, Robert (Scientific Data, 2020)
Canine mammary carcinoma (CMC) has been used as a model to investigate the pathogenesis of human breast cancer and the same grading scheme is commonly used to assess tumor malignancy in both. One key component of this grading scheme is the density of mitotic figures (MF). Current publicly available datasets on human breast cancer only provide annotations for small subsets of whole slide images (WSIs). We present a novel dataset of 21 WSIs of CMC completely annotated for MF. For this, a pathologist screened all WSIs for potential MF and structures with a similar appearance. A second expert blindly assigned labels, and for non-matching labels, a third expert assigned the final labels. Additionally, we used machine learning to identify previously undetected MF. Finally, we performed representation learning and two-dimensional projection to further increase the consistency of the annotations. Our dataset consists of 13,907 MF and 36,379 hard negatives. We achieved a mean F1-score of 0.791 on the test set and of up to 0.696 on a human breast cancer dataset.
CRIS-Team Scopus Importer:2020-12-04

A Completely Annotated Whole Slide Image Dataset of Canine Breast Cancer to Aid Human Breast Cancer Research - Aubreville, Marc; Bertram, Christof A.; Donovan, Taryn A.; Marzahl, Christian; Maier, Andreas; Klopfleisch, Robert (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

A large-scale dataset for mitotic figure assessment on whole slide images of canine cutaneous mast cell tumor - Bertram, Christof A.; Aubreville, Marc; Marzahl, Christian; Maier, Andreas; Klopfleisch, Robert (Scientific Data, 2019)
We introduce a novel, large-scale dataset for microscopy cell annotations. The dataset includes 32 whole slide images (WSI) of canine cutaneous mast cell tumors, selected to include both low grade cases as well as high grade cases. The slides have been completely annotated for mitotic figures and we provide secondary annotations for neoplastic mast cells, inflammatory granulocytes, and mitotic figure look-alikes. Additionally to a blinded two-expert manual annotation with consensus, we provide an algorithm-aided dataset, where potentially missed mitotic figures were detected by a deep neural network and subsequently assessed by two human experts. We included 262,481 annotations in total, out of which 44,880 represent mitotic figures. For algorithmic validation, we used a customized RetinaNet approach, followed by a cell classification network. We find F1-Scores of 0.786 and 0.820 for the manually labelled and the algorithm-aided dataset, respectively. The dataset provides, for the first time, WSIs completely annotated for mitotic figures and thus enables assessment of mitosis detection algorithms on complete WSIs as well as region of interest detection algorithms.
Keywords: mitotic figure detection; tumor grading; digital pathology

Are fast labeling methods reliable? a case study of computer-aided expert annotations on microscopy slides - Marzahl, Christian; Bertram, Christof A.; Aubreville, Marc; Petrick, Anne; Weiler, Kristina; Gläsel, Agnes C.; Fragoso, Marco; Merz, Sophie; Bartenschlager, Florian; Hoppe, Judith; Langenhagen, Alina; Jasensky, Anne Katherine; et al. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020)
Deep-learning-based pipelines have shown the potential to revolutionalize microscopy image diagnostics by providing visual augmentations and evaluations to a trained pathology expert. However, to match human performance, the methods rely on the availability of vast amounts of high-quality labeled data, which poses a significant challenge. To circumvent this, augmented labeling methods, also known as expert-algorithm-collaboration, have recently become popular. However, potential biases introduced by this operation mode and their effects for training deep neuronal networks are not entirely understood. This work aims to shed light on some of the effects by providing a case study for three pathologically relevant diagnostic settings. Ten trained pathology experts performed a labeling tasks first without and later with computer-generated augmentation. To investigate different biasing effects, we intentionally introduced errors to the augmentation. In total, the pathology experts annotated 26,015 cells on 1,200 images in this novel annotation study. Backed by this extensive data set, we found that the concordance of multiple experts was significantly increased in the computer-aided setting, versus the unaided annotation. However, a significant percentage of the deliberately introduced false labels was not identified by the experts.
CRIS-Team Scopus Importer:2020-10-30
Keywords: Computer-aided labelling; Expert-algorithm collaboration; Microscopy; Pathology

Are Pathologist-Defined Labels Reproducible? Comparison of the TUPAC16 Mitotic Figure Dataset with an Alternative Set of Labels - Bertram, Christof A.; Veta, Mitko; Marzahl, Christian; Stathonikos, Nikolas; Maier, Andreas; Klopfleisch, Robert; Aubreville, Marc (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020)
Pathologist-defined labels are the gold standard for histopathological data sets, regardless of well-known limitations in consistency for some tasks. To date, some datasets on mitotic figures are available and were used for development of promising deep learning-based algorithms. In order to assess robustness of those algorithms and reproducibility of their methods it is necessary to test on several independent datasets. The influence of different labeling methods of these available datasets is currently unknown. To tackle this, we present an alternative set of labels for the images of the auxiliary mitosis dataset of the TUPAC16 challenge. Additional to manual mitotic figure screening, we used a novel, algorithm-aided labeling process, that allowed to minimize the risk of missing rare mitotic figures in the images. All potential mitotic figures were independently assessed by two pathologists. The novel, publicly available set of labels contains 1,999 mitotic figures (+28.80%) and additionally includes 10,483 labels of cells with high similarities to mitotic figures (hard examples). We found significant difference comparing scores between the original label set (0.549) and the new alternative label set (0.735) using a standard deep learning object detection architecture. The models trained on the alternative set showed higher overall confidence values, suggesting a higher overall label consistency. Findings of the present study show that pathologists-defined labels may vary significantly resulting in notable difference in the model performance. Comparison of deep learning-based algorithms between independent datasets with different labeling methods should be done with caution.
CRIS-Team Scopus Importer:2020-10-30
Keywords: Breast cancer; Computer-aided annotation; Deep learning; Mitotic figures

Automated diagnosis of 7 canine skin tumors using machine learning on H&E-stained whole slide images - Fragoso-Garcia, Marco; Wilm, Frauke; Bertram, Christof A.; Merz, Sophie; Schmidt, Anja; Donovan, Taryn; Fuchs-Baumgartinger, Andrea; Bartel, Alexander; Marzahl, Christian; Diehl, Laura; Puget, Chloe; Maier, Andreas; et al. (Veterinary Pathology, 2023)
Microscopic evaluation of hematoxylin and eosin-stained slides is still the diagnostic gold standard for a variety of diseases, including neoplasms. Nevertheless, intra- and interrater variability are well documented among pathologists. So far, computer assistance via automated image analysis has shown potential to support pathologists in improving accuracy and reproducibility of quantitative tasks. In this proof of principle study, we describe a machine-learning-based algorithm for the automated diagnosis of 7 of the most common canine skin tumors: trichoblastoma, squamous cell carcinoma, peripheral nerve sheath tumor, melanoma, histiocytoma, mast cell tumor, and plasmacytoma. We selected, digitized, and annotated 350 hematoxylin and eosin-stained slides (50 per tumor type) to create a database divided into training, n = 245 whole-slide images (WSIs), validation (n = 35 WSIs), and test sets (n = 70 WSIs). Full annotations included the 7 tumor classes and 6 normal skin structures. The data set was used to train a convolutional neural network (CNN) for the automatic segmentation of tumor and nontumor classes. Subsequently, the detected tumor regions were classified patch-wise into 1 of the 7 tumor classes. A majority of patches-approach led to a tumor classification accuracy of the network on the slide-level of 95% (133/140 WSIs), with a patch-level precision of 85%. The same 140 WSIs were provided to 6 experienced pathologists for diagnosis, who achieved a similar slide-level accuracy of 98% (137/140 correct majority votes). Our results highlight the feasibility of artificial intelligence-based methods as a support tool in diagnostic oncologic pathology with future applications in other species and tumor types.
CRIS-Team Scopus Importer:2023-08-11
Keywords: computational pathology; computer-aided diagnosis; digital pathology; dog; machine learning; skin; veterinary oncology

Cell Detection for Asthma on Partially Annotated Whole Slide Images: Learning to be EXACT - Marzahl, Christian; Bertram, Christof A.; Wilm, Frauke; Voigt, Jörn; Barton, Ann K.; Klopfleisch, Robert; Breininger, Katharina; Maier, Andreas; Aubreville, Marc (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

Classification of leukemic B-Lymphoblast cells from blood smear microscopic images with an attention-based deep learning method and advanced augmentation techniques - Marzahl, Christian; Aubreville, Marc; Voigt, Jörn; Maier, Andreas (ISBI 2019 C-NMC Challenge: Classification in Cancer Cell Imaging, 2019)
Acute Lymphoblastic Leukemia (ALL) is a blood cell cancer type characterized by an increased number of immature lymphocytes. It is one of the leading cancer inflicted causes of death among children and affected around 876.000 people globally in 2015. In this work, we describe methods used for the “Classification of Normal versus Malignant Cells in B-ALL White Blood Cancer Microscopic Images” ISBI challenge. Due to the morphological similarity between malignant and normal cells, the classification of these cell types is a very challenging task, even for human experts. Deep learning using Convolutional Neural Networks (CNN) provides state-of-the-art techniques for image classification, even with limited annotation data, based on techniques like transfer learning and data augmentation. Our solution to tackle this classification problem is based upon advanced augmentation techniques to counter overfitting the small dataset and transfer learning with adaptive learning rates. Additionally, we incorporated a basic attention mechanism based on a region proposal subnetwork to boost our results for this competition. The ResNet 18 network with an additional regression head achieved a weighted F1 score of 0.8284 on the final test set and 0.8746 on the preliminary test set.
CRIS-Team Scopus Importer:2020-02-21
Keywords: Acute Lymphoblastic Leukemia; Attention modeling; Deep learning

Computer Assistance: A New Gold Standard for the Mitotic Count? - Bertram, Christof A.; Aubreville, Marc; Donovan, Taryn A.; Bartel, Alexander; Wilm, Frauke; Marzahl, Christian; Assenmacher, Charles-Antoine; Becker, Kathrin; Bennett, Mark; Corner, Sarah; Cossic, Brieuc; Denk, Daniela; et al. (Journal of Comparative Pathology, 2022)

Computer-assisted mitotic count using a deep learning–based algorithm improves interobserver reproducibility and accuracy - Bertram, Christof A.; Aubreville, Marc; Donovan, Taryn A.; Bartel, Alexander; Wilm, Frauke; Marzahl, Christian; Assenmacher, Charles-Antoine; Becker, Kathrin; Bennett, Mark; Corner, Sarah; Cossic, Brieuc; Denk, Daniela; et al. (Veterinary Pathology, 2022)
The mitotic count (MC) is an important histological parameter for prognostication of malignant neoplasms. However, it has inter- and intraobserver discrepancies due to difficulties in selecting the region of interest (MC-ROI) and in identifying or classifying mitotic figures (MFs). Recent progress in the field of artificial intelligence has allowed the development of high-performance algorithms that may improve standardization of the MC. As algorithmic predictions are not flawless, computer-assisted review by pathologists may ensure reliability. In the present study, we compared partial (MC-ROI preselection) and full (additional visualization of MF candidates and display of algorithmic confidence values) computer-assisted MC analysis to the routine (unaided) MC analysis by 23 pathologists for whole-slide images of 50 canine cutaneous mast cell tumors (ccMCTs). Algorithmic predictions aimed to assist pathologists in detecting mitotic hotspot locations, reducing omission of MFs, and improving classification against imposters. The interobserver consistency for the MC significantly increased with computer assistance (interobserver correlation coefficient, ICC = 0.92) compared to the unaided approach (ICC = 0.70). Classification into prognostic stratifications had a higher accuracy with computer assistance. The algorithmically preselected hotspot MC-ROIs had a consistently higher MCs than the manually selected MC-ROIs. Compared to a ground truth (developed with immunohistochemistry for phosphohistone H3), pathologist performance in detecting individual MF was augmented when using computer assistance (F1-score of 0.68 increased to 0.79) with a reduction in false negatives by 38%. The results of this study demonstrate that computer assistance may lead to more reproducible and accurate MCs in ccMCTs.
CRIS-Team Scopus Importer:2022-03-25
Keywords: artificial intelligence; automated image analysis; canine cutaneous mast cell tumors; computer assistance; deep learning; digital pathology; mitotic count; mitotic figures

Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm - Bertram, Christof A.; Marzahl, Christian; Bartel, Alexander; Stayt, Jason; Bonsembiante, Federico; Beeler-Marfisi, Janet; Barton, Ann K.; Brocca, Ginevra; Gelain, Maria E.; Glasel, Agnes; Du Preez, Kelly; Weiler, Kristina; et al. (Veterinary Pathology, 2022)
Exercise-induced pulmonary hemorrhage (EIPH) is a relevant respiratory disease in sport horses, which can be diagnosed by examination of bronchoalveolar lavage fluid (BALF) cells using the total hemosiderin score (THS). The aim of this study was to evaluate the diagnostic accuracy and reproducibility of annotators and to validate a deep learning-based algorithm for the THS. Digitized cytological specimens stained for iron were prepared from 52 equine BALF samples. Ten annotators produced a THS for each slide according to published methods. The reference methods for comparing annotator’s and algorithmic performance included a ground truth dataset, the mean annotators’ THSs, and chemical iron measurements. Results of the study showed that annotators had marked interobserver variability of the THS, which was mostly due to a systematic error between annotators in grading the intracytoplasmatic hemosiderin content of individual macrophages. Regarding overall measurement error between the annotators, 87.7% of the variance could be reduced by using standardized grades based on the ground truth. The algorithm was highly consistent with the ground truth in assigning hemosiderin grades. Compared with the ground truth THS, annotators had an accuracy of diagnosing EIPH (THS of < or ≥ 75) of 75.7%, whereas, the algorithm had an accuracy of 92.3% with no relevant differences in correlation with chemical iron measurements. The results show that deep learning-based algorithms are useful for improving reproducibility and routine applicability of the THS. For THS by experts, a diagnostic uncertainty interval of 40 to 110 is proposed. THSs within this interval have insufficient reproducibility regarding the EIPH diagnosis.
CRIS-Team Scopus Importer:2022-12-02
Keywords: artificial intelligence; automated image analysis; bronchoalveolar lavage fluid; computational pathology; digital pathology; equine; pulmonary hemorrhage; respiratory disease; total hemosiderin score

Dataset on Bi- and Multi-nucleated Tumor Cells in Canine Cutaneous Mast Cell Tumors - Bertram, Christof A.; Donovan, Taryn A.; Tecilla, Marco; Bartenschlager, Florian; Fragoso, Marco; Wilm, Frauke; Marzahl, Christian; Breininger, Katharina; Maier, Andreas; Klopfleisch, Robert; Aubreville, Marc (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region - Aubreville, Marc; Bertram, Christof A.; Marzahl, Christian; Gurtner, Corinne; Dettwiler, Martina; Schmidt, Anja; Bartenschlager, Florian; Merz, Sophie; Fragoso, Marco; Kershaw, Olivia; Klopfleisch, Robert; Maier, Andreas (Scientific Reports, 2020)
Manual count of mitotic figures, which is determined in the tumor region with the highest mitotic activity, is a key parameter of most tumor grading schemes. It can be, however, strongly dependent on the area selection due to uneven mitotic figure distribution in the tumor section. We aimed to assess the question, how significantly the area selection could impact the mitotic count, which has a known high inter-rater disagreement. On a data set of 32 whole slide images of H&E-stained canine cutaneous mast cell tumor, fully annotated for mitotic figures, we asked eight veterinary pathologists (five board-certified, three in training) to select a field of interest for the mitotic count. To assess the potential difference on the mitotic count, we compared the mitotic count of the selected regions to the overall distribution on the slide. Additionally, we evaluated three deep learning-based methods for the assessment of highest mitotic density: In one approach, the model would directly try to predict the mitotic count for the presented image patches as a regression task. The second method aims at deriving a segmentation mask for mitotic figures, which is then used to obtain a mitotic density. Finally, we evaluated a two-stage object-detection pipeline based on state-of-the-art architectures to identify individual mitotic figures. We found that the predictions by all models were, on average, better than those of the experts. The two-stage object detector performed best and outperformed most of the human pathologists on the majority of tumor cases. The correlation between the predicted and the ground truth mitotic count was also best for this approach (0.963–0.979). Further, we found considerable differences in position selection between pathologists, which could partially explain the high variance that has been reported for the manual mitotic count. To achieve better inter-rater agreement, we propose to use a computer-based area selection for support of the pathologist in the manual mitotic count.
CRIS-Team Scopus Importer:2020-10-16

Deep Learning-based Cell Detection in Microscopy Images - Marzahl, Christian (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2022)
In recent years, classical pathology has been in a constant state of change towards digital pathology. Digitisation forms the basis for supporting the work of pathologists in many ways by utilising computer-assisted methods. Among other things, modern methods of pattern recognition can accelerate previously time-consuming quantitative analyses, support reporting, and thus standardise and improve the results of pathological examinations. Digitalisation also offers new possibilities and ways of interdisciplinary cooperation and coordination between experts at an international level. This work presents novel methods and software that can be used to support both the analysis of image data and the cooperation between users. For these purposes, species-independent cell detection methods are developed and analysed, allowing the automatic quantification of pulmonary haemorrhage and asthma on cytological image data. The basis for developing these methods is qualitatively, and quantitatively comprehensive data sets created online and interdisciplinary with the annotation tool EXACT developed in this thesis. For cell detection, an established deep learning-based method of object recognition is optimised regarding resource consumption and training pipeline to meet the specific requirements of pathological problems such as image size and unbalanced annotation distribution. Employing the developed detection methods, algorithmically, pre-labelled images can be presented to the annotators for verification and thus, can efficiently increase existing data sets in quantity and quality. In this context, the thesis also investigates whether humans adopt algorithmic errors. The generalisation of algorithms across species and scanner boundaries is of great scientific and economic interest. Therefore, as an initial step for this dissertation, a dataset with human, feline and horse cell samples on the question of pulmonary haemorrhage is created which is the first of its kind in terms of both the number of annotated cells and the number of species. A registration method is developed in a second step that abstracts the pyramid representation of pathological images using a quadtree. In contrast to established methods, this approach does not require pre-segmentation or images with lower resolution and is the only one that can be applied natively to images with cytological questions. The findings and algorithms from the developments and studies described in this thesis are incorporated into the realisation of the online annotation tool EXACT in close cooperation with interdisciplinary experts. In contrast to established tools, EXACT places its focus on the traceability of annotation creation and various visualisation techniques to increase the quality of annotations. The presented methods for object recognition achieve accuracy that is comparable to the one achieved by human expert while at the same time increase the efficiency and reproducibility of grading cytological images. Thus, these methods form the basis for creating and publishing qualitatively and quantitatively comprehensive multi-species datasets using the annotation tool developed in this thesis. While the routine use of the developed algorithms still requires extensive cross-institutional studies that independently prove the added value of these procedures, the online annotation tool EXACT is already successfully used in several companies and research projects for the creation and registration of a broad range of dat

Deep Learning-Based Quantification of Pulmonary Hemosiderophages in Cytology Slides - Marzahl, Christian; Aubreville, Marc; Bertram, Christof A.; Stayt, Jason; Jasensky, Anne Katherine; Bartenschlager, Florian; Fragoso-Garcia, Marco; Barton, Ann K.; Elsemann, Svenja; Jabari, Samir; Krauth, Jens; Madhu, Prathmesh; et al. (Scientific Reports, 2020)
Exercise-induced pulmonary hemorrhage (EIPH) is a common condition in sport horses with negative impact on performance. Cytology of bronchoalveolar lavage fluid by use of a scoring system is considered the most sensitive diagnostic method. Macrophages are classified depending on the degree of cytoplasmic hemosiderin content. The current gold standard is manual grading, which is however monotonous and time-consuming. We evaluated state-of-the-art deep learning-based methods for single cell macrophage classification and compared them against the performance of nine cytology experts and evaluated inter- and intra-observer variability. Additionally, we evaluated object detection methods on a novel data set of 17 completely annotated cytology whole slide images (WSI) containing 78,047 hemosiderophages. Our deep learning-based approach reached a concordance of 0.85, partially exceeding human expert concordance (0.68 to 0.86, mean of 0.73, SD of 0.04). Intra-observer variability was high (0.68 to 0.88) and inter-observer concordance was moderate (Fleiss’ kappa = 0.67). Our object detection approach has a mean average precision of 0.66 over the five classes from the whole slide gigapixel image and a computation time of below two minutes. To mitigate the high inter- and intra-rater variability, we propose our automated object detection pipeline, enabling accurate, reproducible and quick EIPH scoring in WSI.
CRIS-Team Scopus Importer:2020-08-14

Domain Adversarial RetinaNet as a Reference Algorithm for the MItosis DOmain Generalization Challenge - Wilm, Frauke; Marzahl, Christian; Breininger, Katharina; Aubreville, Marc (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2022)
Assessing the mitotic count has a known high degree of intra- and inter-rater variability. Computer-aided systems have proven to decrease this variability and reduce labeling time. These systems, however, are generally highly dependent on their training domain and show poor applicability to unseen domains. In histopathology, these domain shifts can result from various sources, including different slide scanning systems used to digitize histologic samples. The MItosis DOmain Generalization challenge focused on this specific domain shift for the task of mitotic figure detection. This work presents a mitotic figure detection algorithm developed as a baseline for the challenge, based on domain adversarial training. On the challenge’s test set, the algorithm scored an F 1 score of 0.7183. The corresponding network weights and code for implementing the network are made publicly available.
CRIS-Team Scopus Importer:2022-03-25
Keywords: Domain Shift; Histopathology; MIDOG; Mitotic Count; Object Detection

EXACT: a collaboration toolset for algorithm-aided annotation of images with annotation version control - Marzahl, Christian; Aubreville, Marc; Bertram, Christof A.; Maier, Jennifer; Bergler, Christian; Kröger, Christine; Voigt, Jörn; Breininger, Katharina; Klopfleisch, Robert; Maier, Andreas (Scientific Reports, 2021)
In many research areas, scientific progress is accelerated by multidisciplinary access to image data and their interdisciplinary annotation. However, keeping track of these annotations to ensure a high-quality multi-purpose data set is a challenging and labour intensive task. We developed the open-source online platform EXACT (EXpert Algorithm Collaboration Tool) that enables the collaborative interdisciplinary analysis of images from different domains online and offline. EXACT supports multi-gigapixel medical whole slide images as well as image series with thousands of images. The software utilises a flexible plugin system that can be adapted to diverse applications such as counting mitotic figures with a screening mode, finding false annotations on a novel validation view, or using the latest deep learning image analysis technologies. This is combined with a version control system which makes it possible to keep track of changes in the data sets and, for example, to link the results of deep learning experiments to specific data set versions. EXACT is freely available and has already been successfully applied to a broad range of annotation tasks, including highly diverse applications like deep learning supported cytology scoring, interdisciplinary multi-centre whole slide image tumour annotation, and highly specialised whale sound spectroscopy clustering.
CRIS-Team Scopus Importer:2021-04-09

First Steps on Gamification of Lung Fluid Cells Annotations in the Flower Domain - Kunzmann, Sonja; Marzahl, Christian; Denzinger, Felix; Bertram, Christof; Klopfleisch, Robert; Breininger, Katharina; Christlein, Vincent; Maier, Andreas (Informatik aktuell, 2022)
Annotating data, especially in the medical domain, requires expert knowledge and a lot of effort. This limits the amount and/or usefulness of available medical data sets for experimentation. Therefore, developing strategies to increase the number of annotations while lowering the needed domain knowledge is of interest. A possible strategy is the use of gamification, i.e. transforming the annotation task into a game. We propose an approach to gamify the task of annotating lung fluid cells from pathological whole slide images (WSIs). As the domain is unknown to non-expert annotators, we transform images of cells to the domain of flower images using a CycleGAN architecture. In this more assessable domain, non-expert annotators can be (t)asked to annotate different kinds of flowers in a playful setting. In order to provide a proof of concept, this work shows that the domain transfer is possible by evaluating an image classification network trained on real cell images and tested on the cell images generated by the CycleGAN network (reconstructed cell images) as well as real cell images. The classification network reaches an average accuracy of 94.73% on the original lung fluid cells and 95.25% on the transformed lung fluid cells, respectively. Our study lays the foundation for future research on gamification using CycleGANs.<br
CRIS-Team Scopus Importer:2022-05-06

How Big is Big Enough?: A Large-Scale Histological Dataset of Mitotic Figures - Bertram, Christof A.; Aubreville, Marc; Marzahl, Christian; Maier, Andreas; Klopfleisch, Robert (Bildverarbeitung für die Medizin 2020, 2020)
Quantification of mitotic figures (MF) within the tumor areas of highest mi- totic density is the most important prognostic parameter for outcome assessment of many tumor types. However, high intra- and inter-rater variability results from difficulties in individual MF identification and region of interest (ROI) se- lection due to uneven MF distribution. Deep learning-based algorithms for MF detection and ROI selection are very promising methods to overcome these lim- itations. As of today, few datasets of human mammary carcinoma are available. They provide labels only in small image sections of the whole slide image (WSI) and include up to 1,552 MF annotations. Our research group has developed a large-scale, open access dataset with annotations for MF in 32 cases of canine cutaneous mast cell tumors. Entire WSI were completely labeled by two pathologists resulting in 44,800 MF anno- tations. Of those, 5.5% were initially missed by expert WSI screening and added through a deep learning-based pipeline for identification of potential candidates. For algorithmic validation, we used a two-stage approach (RetinaNet followed by cell classificator), which yielded a F1 score of 0.820. Through the algorith- m-aided completion of the dataset we were able to increase the F1 score by 3.4 percentage points. Influence of the size of the dataset was assessed by stepwise reduction of the number of WSI and size (in high power fields, HPF) of the image sections used for training. With the number of included images, the F1 score moderately increased (3 WSI: 0.772; 6 WSI: 0.804; 12 WSI: 0.817; 21 WSI: 0.820). The size of the tumor area in training (ROI selected by an expert) had significant effects on the F1 score (5 HPF: 0.583; 10 HPF: 0.676; 50 HPF: 0.770; complete WSI: 0.820), which was determined in entire WSI of the test set. We emphasize the benefit of appropriate dataset size and complete WSI labeling.
Keywords: mitotic figures; tumor diagnostics; open data; big data; canine cutaneous mast cell tumor

Influence of Inter-Annotator Variability on Automatic Mitotic Figure Assessment - Wilm, Frauke; Bertram, Christof A.; Marzahl, Christian; Bartel, Alexander; Donovan, Taryn A.; Assenmacher, Charles Antoine; Becker, Kathrin; Bennett, Mark; Corner, Sarah; Cossic, Brieuc; Denk, Daniela; Dettwiler, Martina; et al. (Informatik aktuell, 2021)
CRIS-Team Scopus Importer:2021-04-19

Inter-species cell detection - datasets on pulmonary hemosiderophages in equine, human and feline specimens - Marzahl, Christian; Hill, Jenny; Stayt, Jason; Bienzle, Dorothee; Welker, Lutz; Wilm, Frauke; Voigt, Joern; Aubreville, Marc; Maier, Andreas; Klopfleisch, Robert; Breininger, Katharina; Bertram, Christof A. (Scientific Data, 2022)
Pulmonary hemorrhage (P-Hem) occurs among multiple species and can have various causes. Cytology of bronchoalveolar lavage fluid (BALF) using a 5-tier scoring system of alveolar macrophages based on their hemosiderin content is considered the most sensitive diagnostic method. We introduce a novel, fully annotated multi-species P-Hem dataset, which consists of 74 cytology whole slide images (WSIs) with equine, feline and human samples. To create this high-quality and high-quantity dataset, we developed an annotation pipeline combining human expertise with deep learning and data visualisation techniques. We applied a deep learning-based object detection approach trained on 17 expertly annotated equine WSIs, to the remaining 39 equine, 12 human and 7 feline WSIs. The resulting annotations were semi-automatically screened for errors on multiple types of specialised annotation maps and finally reviewed by a trained pathologist. Our dataset contains a total of 297,383 hemosiderophages classified into five grades. It is one of the largest publicly available WSIs datasets with respect to the number of annotations, the scanned area and the number of species covered.
CRIS-Team WoS Importer:2022-06-17

Inter-Species, Inter-Tissue Domain Adaptation for Mitotic Figure Assessment - Learning New Tricks from Old Dogs - Aubreville, Marc; Bertram, Christof A.; Jabari, Samir; Marzahl, Christian; Klopfleisch, Robert; Maier, Andreas (Bildverarbeitung für die Medizin 2020, 2020)
For histopathological tumor assessment, the count of mitotic figures per area is an important part of prognostication. Algorithmic approaches - such as for mitotic figure identification - have significantly improved in recent times, potentially allowing for computer-augmented or fully automatic screening systems in the future. This trend is further supported by whole slide scanning microscopes becoming available in many pathology labs and could soon become a standard imaging tool. For an application in broader fields of such algorithms, the availability of mitotic figure data sets of sufficient size for the respective tissue type and species is an important precondition, that is, however, rarely met. While algorithmic performance climbed steadily for e.g. human mammary carcinoma, thanks to several challenges held in the field, for many tumor types, data sets are not available. In this work, we assess domain transfer of mitotic figure recognition using domain adversarial training on four data sets, two from dogs and two from humans. We were able to show that domain adversarial training considerably improves accuracy when applying mitotic _gure classification learned from the canine on the human data sets (up to +12.8% in accuracy) and is thus a helpful method to transfer knowledge from existing data sets to new tissue types and species.<br
Keywords: unsupervised domain adaptation; mitotic figure; tumor diagnostic; deep neural networks

Is crowd-algorithm collaboration an advanced alternative to crowd-sourcing on cytology slides? - Marzahl, Christian; Aubreville, Marc; Bertram, Christof A.; Gerlach, Stefan; Maier, Jennifer; Voigt, Jörn; Hill, Jenny; Klopfleisch, Robert; Maier, Andreas (Informatik aktuell, 2020)
Modern, state-of-the-art deep learning approaches yield human like performance in numerous object detection and classification tasks. The foundation for their success is the availability of training datasets of substantially high quantity, which are expensive to create, especially in the field of medical imaging. Crowdsourcing has been applied to create large datasets for a broad range of disciplines. This study aims to explore the challenges and opportunities of crowd-algorithm collaboration for the object detection task of grading cytology whole slide images. We compared the classical crowdsourcing performance of twenty participants with their results from crowd-algorithm collaboration. All participants performed both modes in random order on the same twenty images. Additionally, we introduced artificial systematic flaws into the precomputed annotations to estimate a bias towards accepting precomputed annotations. We gathered 9524 annotations on 800 images from twenty participants organised into four groups in concordance to their level of expertise with cytology. The crowd-algorithm mode improved on average the participants’ classification accuracy by 7%, the mean average precision by 8% and the inter-observer Fleiss’ kappa score by 20%, and reduced the time spent by 31%. However, two thirds of the artificially modified false labels were not recognised as such by the contributors. This study shows that crowd-algorithm collaboration is a promising new approach to generate large datasets when it is ensured that a carefully designed setup eliminates potential biases.
CRIS-Team Scopus Importer:2020-04-21

Iterative Cross-Scanner Registration for Whole Slide Images - Theelke, Luisa; Wilm, Frauke; Marzahl, Christian; Bertram, Christof A.; Klopfleisch, Robert; Maier, Andreas; Aubreville, Marc; Breininger, Katharina (2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021), 2021)
The successful registration of digitized microscopic images is required for many applications in digital pathology. In particular, the registration of specimens scanned by different slide scanning systems may be beneficial to transfer expert annotations from one image domain to another and thereby reduce labeling effort. We present an iterative approach to register microscopic specimens digitized with multiple scanning systems, aiming to compute an optimal global transformation for the images at highest resolution. For this purpose, an initial registration based on a down-scaled version of the images is followed by a patch-based iterative update scheme. We make use of the hierarchical structure of digitized whole slide images to gradually approximate the optimal transformation. By using kernel density estimation to weight local transformation estimates, the influence of registration errors can be further mitigated. We validate our method on five histologic and five cytologic samples, each scanned with four different scanning systems. Furthermore, we perform first experiments on samples stained with different stain combinations. Our experiments demonstrate the potential of the proposed method for a variety of datasets and application fields.<br
CRIS-Team WoS Importer:2022-02-25

Mitosis domain generalization in histopathology images — The MIDOG challenge - Aubreville, Marc; Stathonikos, Nikolas; Bertram, Christof A.; Klopfleisch, Robert; ter Hoeve, Natalie; Ciompi, Francesco; Wilm, Frauke; Marzahl, Christian; Donovan, Taryn A.; Maier, Andreas; Breen, Jack; Ravikumar, Nishant; et al. (Medical Image Analysis, 2023)
The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate when applied in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MIDOG 2021 challenge was the creation of scanner-agnostic MF detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were provided. In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance. The winning algorithm yielded an F1 score of 0.748 (CI95: 0.704-0.781), exceeding the performance of six experts on the same task.
CRIS-Team Scopus Importer:2022-12-16
Keywords: Challenge; Deep Learning; Domain generalization; Histopathology; Mitosis

Pan-tumor CAnine cuTaneous Cancer Histology (CATCH) dataset - Wilm, Frauke; Fragoso, Marco; Marzahl, Christian; Qiu, Jingna; Puget, Chloe; Diehl, Laura; Bertram, Christof A.; Klopfleisch, Robert; Maier, Andreas; Breininger, Katharina; Aubreville, Marc (Scientific Data, 2022)
Due to morphological similarities, the differentiation of histologic sections of cutaneous tumors into individual subtypes can be challenging. Recently, deep learning-based approaches have proven their potential for supporting pathologists in this regard. However, many of these supervised algorithms require a large amount of annotated data for robust development. We present a publicly available dataset of 350 whole slide images of seven different canine cutaneous tumors complemented by 12,424 polygon annotations for 13 histologic classes, including seven cutaneous tumor subtypes. In inter-rater experiments, we show a high consistency of the provided labels, especially for tumor annotations. We further validate the dataset by training a deep neural network for the task of tissue segmentation and tumor subtype classification. We achieve a class-averaged Jaccard coefficient of 0.7047, and 0.9044 for tumor in particular. For classification, we achieve a slide-level accuracy of 0.9857. Since canine cutaneous tumors possess various histologic homologies to human tumors the added value of this dataset is not limited to veterinary pathology but extends to more general fields of application.
CRIS-Team Scopus Importer:2022-10-14

Robust Quad-Tree based Registration on Whole Slide Images - Marzahl, Christian; Wilm, Frauke; Dressler, Franz F.; Tharun, Lars; Perner, Sven; Bertram, Christof A.; Kröger, Christine; Voigt, Jörn; Klopfleisch, Robert; Maier, Andreas; Aubreville, Marc; Breininger, Katharina (MICCAI Computational Pathlogy (COMPAY) Workshop, 2021)
The registration of whole slide images (WSIs) provides the basis for many subsequent processing steps in digital pathology. For instance, the registration of immunohistochemistry (IHC) and hematoxylin & eosin (H&E)-stained WSIs is usually the first step in guiding IHC diagnostic procedures. Still, many registration methods operate poorly on WSIs. Reasons for this include the WSI size, fluctuating image quality or elastic tissue deformations. Multiple prior methods are further specialised towards a specific image modality, such as histology or cytology, or rely on a specific preparation protocol. To minimise these effects, we developed a robust WSI registration, which differs from previous methods by the following new aspect: We introduce a multi-scale approach based on a quad-tree (QT), with several termination criteria that makes the algorithm particularly insensitive to tissue artefacts and that further allows to estimate a piece-wise affine transformation. We validated our method on five scanner systems and 60 WSIs with different stainings. Our results outperformed any publicly available WSI registration method. The QT code, WSI landmarks and tools used to create the validation dataset are made publicly available.<br

Superpixel Pre-segmentation of HER2 Slides for Efficient Annotation - Öttl, Mathias; Mönius, Jana; Marzahl, Christian; Rübner, Matthias; Geppert, Carol-Immanuel; Hartmann, Arndt; Beckmann, Matthias; Fasching, Peter; Maier, Andreas; Erber, Ramona; Breininger, Katharina (Informatik aktuell, 2022)
Supervised deep learning has shown state-of-the-art performance for medical image segmentation across different applications, including histopathology and cancer research; however, the manual annotation of such data is extremely laborious. In this work, we explore the use of superpixel approaches to compute a pre-segmentation of HER2 stained images for breast cancer diagnosis that facilitates faster manual annotation and correction in a second step. Four methods are compared: standard simple linear iterative clustering (SLIC) as a baseline, a domain adapted SLIC, and superpixels based on feature embeddings of a pretrained ResNet-50 and a denoising autoencoder. To tackle oversegmentation, we propose to hierarchically merge superpixels, based on their content in the respective feature space. When evaluating the approaches on fully manually annotated images, we observe that the autoencoder-based superpixels achieve a 23% increase in boundary F1 score compared to the baseline SLIC superpixels. Furthermore, the boundary F1 score increases by 73% when hierarchical clustering is applied on the adapted SLIC and the autoencoder-based superpixels. These evaluations show encouraging first results for a pre-segmentation for efficient manual refinement without the need for an initial set of annotated training data.<br
CRIS-Team Scopus Importer:2022-05-06