Hagag A, Gomaa A, Kornek D, Maier A, Fietkau R, Bert C, Huang Y, Putz F (2024)
Publication Type: Conference contribution
Publication year: 2024
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 15005 LNCS
Pages Range: 198-208
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Marrakesh, MAR
ISBN: 9783031720857
DOI: 10.1007/978-3-031-72086-4_19
Survival prediction for cancer patients is critical for optimal treatment selection and patient management. Current patient survival prediction methods typically extract survival information from patients’ clinical record data or biological and imaging data. In practice, experienced clinicians can have a preliminary assessment of patients’ health status based on patients’ observable physical appearances, which are mainly facial features. However, such assessment is highly subjective. In this work, the efficacy of objectively capturing and using prognostic information contained in conventional portrait photographs using deep learning for survival prediction purposes is investigated for the first time. A pre-trained StyleGAN2 model is fine-tuned on a custom dataset of our cancer patients’ photos to empower its generator with generative ability suitable for patients’ photos. The StyleGAN2 is then used to embed the photographs to its highly expressive latent space. Utilizing state-of-the-art survival analysis models and StyleGAN’s latent space embeddings, this approach predicts the overall survival for single as well as pan-cancer, achieving a C-index of 0.680 in a pan-cancer analysis, showcasing the prognostic value embedded in simple 2D facial images. In addition, thanks to StyleGAN’s interpretable latent space, our survival prediction model can be validated for relying on essential facial features, eliminating any biases from extraneous information like clothing or background. Moreover, our approach provides a novel health attribute obtained from StyleGAN’s extracted features, allowing the modification of face photographs to either a healthier or more severe illness appearance, which has significant prognostic value for patient care and societal perception, underscoring its potential important clinical value.
APA:
Hagag, A., Gomaa, A., Kornek, D., Maier, A., Fietkau, R., Bert, C.,... Putz, F. (2024). Deep Learning for Cancer Prognosis Prediction Using Portrait Photos by StyleGAN Embedding. In Marius George Linguraru, Qi Dou, Aasa Feragen, Stamatia Giannarou, Ben Glocker, Karim Lekadir, Julia A. Schnabel (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 198-208). Marrakesh, MAR: Springer Science and Business Media Deutschland GmbH.
MLA:
Hagag, Amr, et al. "Deep Learning for Cancer Prognosis Prediction Using Portrait Photos by StyleGAN Embedding." Proceedings of the 27th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2024, Marrakesh, MAR Ed. Marius George Linguraru, Qi Dou, Aasa Feragen, Stamatia Giannarou, Ben Glocker, Karim Lekadir, Julia A. Schnabel, Springer Science and Business Media Deutschland GmbH, 2024. 198-208.
BibTeX: Download