Spatial transcriptomics expression prediction from histopathology based on cross-modal mask reconstruction and contrastive learning

Liu J, Eckstein M, Wang Z, Feuerhake F, Merhof D (2026)

Publication Type: Journal article

Publication year: 2026

Journal

Medical Image Analysis Elsevier

Book Volume: 108

Article Number: 103889

DOI: 10.1016/j.media.2025.103889

Abstract

Spatial transcriptomics is a technology that captures gene expression at different spatial locations, widely used in tumor microenvironment analysis and molecular profiling of histopathology, providing valuable insights into resolving gene expression and clinical diagnosis of cancer. Due to the high cost of data acquisition, large-scale spatial transcriptomics data remain challenging to obtain. In this study, we develop a contrastive learning-based deep learning method to predict spatially resolved gene expression from the whole-slide images (WSIs). Unlike existing end-to-end prediction frameworks, our method leverages multi-modal contrastive learning to establish a correspondence between histopathological morphology and spatial gene expression in the feature space. By computing cross-modal feature similarity, our method generates spatially resolved gene expression directly from WSIs. Furthermore, to enhance the standard contrastive learning paradigm, a cross-modal masked reconstruction is designed as a pretext task, enabling feature-level fusion between modalities. Notably, our method does not rely on large-scale pretraining datasets or abstract semantic representations from either modality, making it particularly effective for scenarios with limited spatial transcriptomics data. Evaluation across six different disease datasets demonstrates that, compared to existing studies, our method improves Pearson Correlation Coefficient (PCC) in the prediction of highly expressed genes, highly variable genes, and marker genes by 6.27 %, 6.11 %, and 11.26 % respectively. Further analysis indicates that our method preserves gene-gene correlations and applies to datasets with limited samples. Additionally, our method exhibits potential in cancer tissue localization based on biomarker expression. The code repository for this work is available at https://github.com/ngfufdrdh/CMRCNet .

Authors with CRIS profile

Markus Eckstein Institute of Pathology

Involved external institutions

Universität Regensburg DE Germany (DE) Beijing Friendship Hospital / 北京友谊医院 CN China (CN) Medizinische Hochschule Hannover (MHH) / Hannover Medical School DE Germany (DE)

How to cite

APA:

Liu, J., Eckstein, M., Wang, Z., Feuerhake, F., & Merhof, D. (2026). Spatial transcriptomics expression prediction from histopathology based on cross-modal mask reconstruction and contrastive learning. Medical Image Analysis, 108. https://doi.org/10.1016/j.media.2025.103889

MLA:

Liu, Junzhuo, et al. "Spatial transcriptomics expression prediction from histopathology based on cross-modal mask reconstruction and contrastive learning." Medical Image Analysis 108 (2026).

BibTeX: Download