Attention-guided Erasing - Bhandary Panambur, Adarsh; Yu, Hui; Bhat, Sheethal; Madhu, Prathmesh; Bayer, Siming; Maier, Andreas (Bildverarbeitung für die Medizin 2024, 2024)
The assessment of breast density is crucial in the context of breast cancer screening, especially in populations with a higher percentage of dense breast tissues. This study introduces a novel data augmentation technique termed attention-guided erasing (AGE), devised to enhance the downstream classification of four distinct breast density categories in mammography following the BI-RADS recommendation in the Vietnamese cohort. The proposed method integrates supplementary information during transfer learning, utilizing visual attention maps derived from a vision transformer backbone trained using the self-supervised DINO method. These maps are utilized to erase background regions in the mammogram images, unveiling only the potential areas of dense breast tissues to the network. Through the incorporation of AGE during transfer learning with varying random probabilities, we consistently surpass classification performance compared to scenarios without AGE and the traditional random erasing transformation. We validate our methodology using the publicly available VinDr-Mammo dataset. Specifically, we attain a mean F1-score of 0.5910, outperforming values of 0.5594 and 0.5691 corresponding to scenarios without AGE and with random erasing (RE), respectively. This superiority is further substantiated by t-tests, revealing a p-value of p<0.0001, underscoring the statistical significance of our approach.<br

Attention-guided erasing for enhanced transfer learning in breast abnormality classification - Bhandary Panambur, Adarsh; Bhat, Sheethal; Yu, Hui; Madhu, Prathmesh; Bayer, Siming; Maier, Andreas (International Journal of Computer Assisted Radiology and Surgery, 2025)
Purpose Breast cancer remains one of the most prevalent cancers globally, necessitating effective early screening and diagnosis. This study investigates the effectiveness and generalizability of our recently proposed data augmentation technique, attention-guided erasing (AGE), across various transfer learning classification tasks for breast abnormality classification in mammography. Methods AGE utilizes attention head visualizations from DINO self-supervised pretraining to weakly localize regions of interest (ROI) in images. These localizations are then used to stochastically erase non-essential background information from training images during transfer learning. Our research evaluates AGE across two image-level and three patch-level classification tasks. The image-level tasks involve breast density categorization in digital mammography (DM) and malignancy classification in contrast-enhanced mammography (CEM). Patch-level tasks include classifying calcifications and masses in scanned film mammography (SFM), as well as malignancy classification of ROIs in CEM. Results AGE significantly boosts classification performance with statistically significant improvements in mean F1-scores across four tasks compared to baselines. Specifically, for image-level classification of breast density in DM and malignancy in CEM, we achieve gains of 2% and 1.5%, respectively. Additionally, for patch-level classification of calcifications in SFM and CEM ROIs, gains of 0.4% and 0.6% are observed, respectively. However, marginal improvement is noted in the mass classification task, indicating the necessity for further optimization in tasks where critical features may be obscured by erasing techniques. Conclusion Our findings underscore the potential of AGE, a dataset- and task-specific augmentation strategy powered by self-supervised learning, to enhance the downstream classification performance of DL models, particularly involving ViTs, in medical imagin

AUCReshaping: improved sensitivity at high-specificity - Bhat, Sheethal; Mansoor, Awais; Georgescu, Bogdan; Bhandary Panambur, Adarsh; Ghesu, Florin C.; Islam, Saahil; Packhäuser, Kai; Rodriguez Salas, Dalia; Grbic, Sasa; Maier, Andreas (Scientific Reports, 2023)
The evaluation of deep-learning (DL) systems typically relies on the Area under the Receiver-Operating-Curve (AU-ROC) as a performance metric. However, AU-ROC, in its holistic form, does not sufficiently consider performance within specific ranges of sensitivity and specificity, which are critical for the intended operational context of the system. Consequently, two systems with identical AU-ROC values can exhibit significantly divergent real-world performance. This issue is particularly pronounced in the context of anomaly detection tasks, a commonly employed application of DL systems across various research domains, including medical imaging, industrial automation, manufacturing, cyber security, fraud detection, and drug research, among others. The challenge arises from the heavy class imbalance in training datasets, with the abnormality class often incurring a considerably higher misclassification cost compared to the normal class. Traditional DL systems address this by adjusting the weighting of the cost function or optimizing for specific points along the ROC curve. While these approaches yield reasonable results in many cases, they do not actively seek to maximize performance for the desired operating point. In this study, we introduce a novel technique known as AUCReshaping, designed to reshape the ROC curve exclusively within the specified sensitivity and specificity range, by optimizing sensitivity at a predetermined specificity level. This reshaping is achieved through an adaptive and iterative boosting mechanism that allows the network to focus on pertinent samples during the learning process. We primarily investigated the impact of AUCReshaping in the context of abnormality detection tasks, specifically in Chest X-Ray (CXR) analysis, followed by breast mammogram and credit card fraud detection tasks. The results reveal a substantial improvement, ranging from 2 to 40%, in sensitivity at high-specificity levels for binary classification tasks.<br
Keywords: Electrical and electronic engineering; Engineering

Enhancing Zero-Shot Learning in Chest X-ray Diagnostics Using BioBERT for Textual Representation - Bharadwaj, Prakhar; Bhat, Sheethal; Maier, Andreas (Bildverarbeitung für die Medizin 2025, 2025)
Accurate diagnosis of pulmonary diseases from chest X-rays remains a challenging task, especially due to the scarcity of labeled data. In this work, we propose an enhanced zero-shot learning framework that integrates BioBERT—a pre-trained model for biomedical text representation—into a contrastive learning pipeline for medical imaging diagnostics. Our method leverages the synergy between textual radiology reports and image data by combining BioBERT with a CLIP-based architecture. To address the lack of textual data in standard datasets, we generate synthetic radiology reports for each pathology, aiming to mimic the complexity of actual clinical descriptions. Through extensive experiments, we demonstrate significant improvements in disease classification accuracy and generalization, particularly in zero-shot inference scenarios. Our analysis also reveals how similarities in pathology descriptions can lead to misclassifications, emphasizing the importance of nuanced textual representations. This work establishes a zero-shot learning method in medical imaging, highlighting the potential of BioBERT in enhancing automated diagnostic systems for chest X-ray
Keywords: Computer aided diagnostics; Zero-shot learning; chest X-rays

Towards robust zero-shot Chest X-Ray (CXR) classification: Exploring data distribution bias in CXR datasets - Bhat, Sheethal; B. Panambur, Adarsh; Mansoor, Awais; Georgescu, Bogdan; Ghesu, Florin C.; Grbic, Sasa; Maier, Andreas (Bildverarbeitung für die Medizin 2025, 2025)
In recent years, unsupervised classification models have become increasingly significant, primarily due to the difficulties associated with data labeling and its costs. This trend is also notable in the field of medical imaging, particularly with Chest X-rays (CXRs). Among the various unsupervised pretraining methodologies, image-text models like CLIP are highlighted for their considerable enhancements in zero-shot classification. In this study, we perform a detailed analysis of CLIP’s performance using multiple large CXR datasets, investigating how the batch size, dataset size, and distribution biases differentially influence outcomes across various findings. In two distinct experiments,we showan average of 3% enhancement in the macro average zero-shot AUC scores when the batch size is increased, and a corresponding 8% improvement for pneumothorax by the addition of a second dataset. For pleural effusion, where performance is nearly saturated and previous changes had little effect, we examine adding weak supervisory meta-labels and image-to-image contrastive loss, achieving an average 1% improvement in zero-shot AUC. Consequently, our work shows incorporating dataset insights, meta-information and contrastive learning strategies enhances the robustness and accuracy of CLIP-CXR for specific finding
Keywords: Computer aided diagnostics; chest X-rays