PANDA: Prototype-Anchored Alignment for Partially Unpaired Multimodal Learning, with Applications to Alzheimer’s MRI and TCGA Pathology

Bhat S, Chowdhury MR, Perez Toro PA, Wunderlich S, Bayer S, Maier A, Bharat RD (2029)

Publication Status: Submitted

Publication Type: Unpublished / Preprint

Future Publication Type: Journal article

Publication year: 2029

Abstract

Multimodal medical prediction often faces incomplete pairing: auxiliary modal ities with complementary signal are available for only a subset of subjects (or none) and cannot be assumed at deployment. We introduce PANDA (Prototype Anchored Data Alignment), a two-stage framework that transfers auxiliary information to a primary-modality model without auxiliary inputs at inference. Stage 1 learns a shared embedding from the paired subset and estimates class prototypes from auxiliary modalities; Stage 2 trains the primary encoder on all subjects using cross-entropy plus alignment to the frozen prototypes. Because supervision is defined at the class-prototype level, PANDA accommodates arbitrary pairing rates, including zero subject overlap. We evaluate PANDA on two applications. On a 1,021-subject multi-scanner ADNI cohort, we perform AD/CN classification with three auxiliary modalities at distinct pairing rates: tabular scores (44.8%), FDG-PET (18.7%), and external handwriting kinematics (0% overlap). Relative to the same-backbone MRI-only baseline, PANDA attains AUC 0.868 ±0.020 (+7.9pp) and reduces 1.5T CN false positives by 24.3pp; on a fully trainable Conv5-FC3 backbone it reaches AUC 0.893 (best overall). A pairing-rate ablation shows that the joint anchor 1 remains within seed noise from 75% to 5% pairing. On TCGA-Lung survival prediction from whole-slide images with RNA-seq as auxiliary data, PANDA improves over WSI-only on 2-year OS (AUC +3.5pp) and Cox PH (C-index +9.0pts) and outperforms full-fusion training, which underperforms WSI-only, while requiring no RNA at inference; wide confidence intervals on this smaller cohort keep the gains below conventional significance. Overall, PANDA provides a deployment-oriented mechanism for leveraging incomplete auxiliary modalities to improve primary-modality prediction.

Authors with CRIS profile

Involved external institutions

National Institute of Mental Health and Neuroscience (NIMHANS) India (IN)

How to cite

APA:

Bhat, S., Chowdhury, M.R., Perez Toro, P.A., Wunderlich, S., Bayer, S., Maier, A., & Bharat, R.D. (2029). PANDA: Prototype-Anchored Alignment for Partially Unpaired Multimodal Learning, with Applications to Alzheimer’s MRI and TCGA Pathology. (Unpublished, Submitted).

MLA:

Bhat, Sheethal, et al. PANDA: Prototype-Anchored Alignment for Partially Unpaired Multimodal Learning, with Applications to Alzheimer’s MRI and TCGA Pathology. Unpublished, Submitted. 2029.

BibTeX: Download