Fully automatic HER2 tissue segmentation for interpretable HER2 scoring - Öttl, Mathias; Steenpaß, Jana; Wilm, Frauke; Qiu, Jingna; Rübner, Matthias; Lang-Schwarz, Corinna; Taverna, Cecilia; Tava, Francesca; Hartmann, Arndt; Hübner, Hanna; Beckmann, Matthias; Fasching, Peter; et al. (Journal of Pathology Informatics, 2025)
Breast cancer is the most common cancer in women, with HER2 (human epidermal growth factor receptor 2) overexpression playing a critical role in regulating cell growth and division. HER2 status, assessed according to established scoring guidelines, offers important information for treatment selection. However, the complexity of the task leads to variability in human rater assessments. In this work, we propose a fully automated, interpretable HER2 scoring pipeline based on pixel-level semantic segmentations, designed to align with clinical guidelines. Using polygon annotations, our method balances annotation effort with the ability to capture fine-grained details and larger structures, such as non-invasive tumor tissue. To enhance HER2 segmentation, we propose the use of a Wasserstein Dice loss to model class relationships, ensuring robust segmentation and HER2 scoring performance. Additionally, based on observations of pathologists' behavior in clinical practice, we propose a calibration step to the scoring rules, which positively impacts the accuracy and consistency of automated HER2 scoring. Our approach achieves an F1 score of 0.832 on HER2 scoring, demonstrating its effectiveness. This work establishes a potent segmentation pipeline that can be further leveraged to analyze HER2 expression in breast cancer tissue.
CRIS-Team Scopus Importer:2025-04-11
Keywords: Deep learning; HER2; HER2 scoring; Histopathology; Semantic segmentation

Improved HER2 Tumor Segmentation with Subtype Balancing Using Deep Generative Networks - Öttl, Mathias; Steenpaß, Jana; Rübner, Matthias; Geppert, Carol-Immanuel; Qiu, Jingna; Wilm, Frauke; Hartmann, Arndt; Beckmann, Matthias; Fasching, Peter; Maier, Andreas; Erber, Ramona; Breininger, Katharina (Proceedings - International Symposium on Biomedical Imaging, 2023)
Tumor segmentation in histopathology images is often complicated by its composition of different histological subtypes and class imbalance. Oversampling subtypes with low prevalence features is not a satisfactory solution since it eventually leads to overfitting. We propose to create synthetic images with semantically-conditioned deep generative networks and to combine subtype-balanced synthetic images with the original dataset to achieve better segmentation performance. We show the suitability of Generative Adversarial Networks (GANs) and especially diffusion models to create realistic images based on subtype-conditioning for the use case of HER2-stained histopathology. Additionally, we show the capability of diffusion models to conditionally inpaint HER2 tumor areas with modified subtypes. Combining the original dataset with the same amount of diffusion-generated images increased the tumor Dice score from 0.833 to 0.854 and almost halved the variance between the HER2 subtype recalls. These results create the basis for more reliable automatic HER2 analysis with lower performance variance between individual HER2 subtypes.
CRIS-Team Scopus Importer:2023-10-06
Keywords: Diffusion Models; Generative Models; HER2; Histopathology; Segmentation; Subtypes

Style-Extracting Diffusion Models for Semi-supervised Histopathology Segmentation - Öttl, Mathias; Wilm, Frauke; Steenpaß, Jana; Qiu, Jingna; Rübner, Matthias; Hartmann, Arndt; Beckmann, Matthias; Fasching, Peter; Maier, Andreas; Erber, Ramona; Kainz, Bernhard; Breininger, Katharina (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2025)
Deep learning-based image generation has seen significant advancements with diffusion models, notably improving the quality of generated images. Despite these developments, generating images with unseen characteristics beneficial for downstream tasks has received limited attention. To bridge this gap, we propose Style-Extracting Diffusion Models, featuring two conditioning mechanisms. Specifically, we utilize 1) a style conditioning mechanism which allows to inject style information of previously unseen images during image generation and 2) a content conditioning which can be targeted to a downstream task, e.g., layout for segmentation. We introduce a trainable style encoder to extract style information from images, and an aggregation block that merges style information from multiple style inputs. This architecture enables the generation of images with unseen styles in a zero-shot manner, by leveraging styles from unseen images, resulting in more diverse generations. In this work, we use the image layout as target condition and first show the capability of our method on a natural image dataset as a proof-of-concept. We further demonstrate its versatility in histopathology, where we combine prior knowledge about tissue composition and unannotated data to create diverse synthetic images with known layouts. This allows us to generate additional synthetic data to train a segmentation network in a semi-supervised fashion. We verify the added value of the generated images by showing improved segmentation results and lower performance variability between patients when synthetic images are included during segmentation training. The code of the method is publicly available at https://github.com/OettlM/STEDM.
Created from Fastlane, Scopus look-up

Superpixel Pre-segmentation of HER2 Slides for Efficient Annotation - Öttl, Mathias; Mönius, Jana; Marzahl, Christian; Rübner, Matthias; Geppert, Carol-Immanuel; Hartmann, Arndt; Beckmann, Matthias; Fasching, Peter; Maier, Andreas; Erber, Ramona; Breininger, Katharina (Informatik aktuell, 2022)
Supervised deep learning has shown state-of-the-art performance for medical image segmentation across different applications, including histopathology and cancer research; however, the manual annotation of such data is extremely laborious. In this work, we explore the use of superpixel approaches to compute a pre-segmentation of HER2 stained images for breast cancer diagnosis that facilitates faster manual annotation and correction in a second step. Four methods are compared: standard simple linear iterative clustering (SLIC) as a baseline, a domain adapted SLIC, and superpixels based on feature embeddings of a pretrained ResNet-50 and a denoising autoencoder. To tackle oversegmentation, we propose to hierarchically merge superpixels, based on their content in the respective feature space. When evaluating the approaches on fully manually annotated images, we observe that the autoencoder-based superpixels achieve a 23% increase in boundary F1 score compared to the baseline SLIC superpixels. Furthermore, the boundary F1 score increases by 73% when hierarchical clustering is applied on the adapted SLIC and the autoencoder-based superpixels. These evaluations show encouraging first results for a pre-segmentation for efficient manual refinement without the need for an initial set of annotated training data.<br
CRIS-Team Scopus Importer:2022-05-06