D’ARTAGNAN: Counterfactual Video Generation
Reynaud H, Vlontzos A, Dombrowski M, Gilligan Lee C, Beqiri A, Leeson P, Kainz B (2022)
Publication Type: Conference contribution
Publication year: 2022
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 13438 LNCS
Pages Range: 599-609
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Singapore
ISBN: 9783031164514
DOI: 10.1007/978-3-031-16452-1_57
Abstract
Causally-enabled machine learning frameworks could help clinicians to identify the best course of treatments by answering counterfactual questions. We explore this path for the case of echocardiograms by looking into the variation of the Left Ventricle Ejection Fraction, the most essential clinical metric gained from these examinations. We combine deep neural networks, twin causal networks and generative adversarial methods for the first time to build D’ARTAGNAN (Deep ARtificial Twin-Architecture GeNerAtive Networks), a novel causal generative model. We demonstrate the soundness of our approach on a synthetic dataset before applying it to cardiac ultrasound videos to answer the question: “What would this echocardiogram look like if the patient had a different ejection fraction?”. To do so, we generate new ultrasound videos, retaining the video style and anatomy of the original patient, while modifying the Ejection Fraction conditioned on a given input. We achieve an SSIM score of 0.79 and an R2 score of 0.51 on the counterfactual videos. Code and models are available at: https://github.com/HReynaud/dartagnan.
Authors with CRIS profile
Hadrian Jean-Francois Reynaud
Mischa Dombrowski Professur für Image Data Exploration and Analysis Bernhard Kainz Professur für Image Data Exploration and Analysis
Mischa Dombrowski Professur für Image Data Exploration and Analysis Bernhard Kainz Professur für Image Data Exploration and Analysis
Involved external institutions
Imperial College London / The Imperial College of Science, Technology and Medicine
United Kingdom (GB)
University College London (UCL)
United Kingdom (GB)
Ultromics Ltd.
United Kingdom (GB)
United Kingdom (GB)
University College London (UCL)
United Kingdom (GB)
Ultromics Ltd.
United Kingdom (GB)
How to cite
APA:
Reynaud, H., Vlontzos, A., Dombrowski, M., Gilligan Lee, C., Beqiri, A., Leeson, P., & Kainz, B. (2022). D’ARTAGNAN: Counterfactual Video Generation. In Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 599-609). Singapore, SG: Springer Science and Business Media Deutschland GmbH.
MLA:
Reynaud, Hadrien, et al. "D’ARTAGNAN: Counterfactual Video Generation." Proceedings of the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022, Singapore Ed. Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li, Springer Science and Business Media Deutschland GmbH, 2022. 599-609.
BibTeX: Download