Trade-Offs in Fine-Tuned Diffusion Models between Accuracy and Interpretability
Dombrowski M, Reynaud H, Müller J, Baugh M, Kainz B (2024)
Publication Language: English
Publication Status: Published
Publication Type: Conference contribution, Conference Contribution
Publication year: 2024
Publisher: AAAI Press
Series: Proceedings of the AAAI Conference on Artificial Intelligence
City/Town: Washington, DC
Book Volume: 38 No. 19
Pages Range: 21037-21045
Conference Proceedings Title: AAAI-24 Special Track Safe, Robust and Responsible AI Track
Event location: Vancouver
ISBN: 978-1-57735-887-9
DOI: 10.1609/aaai.v38i19.30095
Abstract
Recent advancements in diffusion models have significantly impacted the trajectory of generative machine learning re-search, with many adopting the strategy of fine-tuning pre-trained models using domain-specific text-to-image datasets. Notably, this method has been readily employed for medical applications, such as X-ray image synthesis, leveraging the plethora of associated radiology reports. Yet, a prevailing concern is the lack of assurance on whether these models genuinely comprehend their generated content. With the evolution of text conditional image generation, these models have grown potent enough to facilitate object localization scrutiny. Our research underscores this advancement in the critical realm of medical imaging, emphasizing the crucial role of interpretability. We further unravel a consequential trade-off between image fidelity – as gauged by conventional metrics – and model interpretability in generative diffusion models. Specifically, the adoption of learnable text encoders when fine-tuning results in diminished interpretability. Our in-depth exploration uncovers the underlying factors responsible for this divergence. Consequently, we present a set of design principles for the development of truly interpretable generative models. Code is available at https://github.com/MischaD/chest-distillation.
Authors with CRIS profile
Mischa Dombrowski
Professur für Image Data Exploration and Analysis
Hadrian Jean-Francois Reynaud
Johanna Müller Professur für Image Data Exploration and Analysis Matthew Moses George Baugh
Bernhard Kainz Professur für Image Data Exploration and Analysis
Johanna Müller Professur für Image Data Exploration and Analysis Matthew Moses George Baugh
Bernhard Kainz Professur für Image Data Exploration and Analysis
Related research project(s)
Medical Image Analysis with Normative Machine Learning (ERC-CoG MIA-NORMAL)
Sept. 1, 2023 - Sept. 30, 2028
DC-AIDE - Dedizierte klinische Ausrüstung für den Einsatz künstlicher Intelligenz (DFG 512819079)
Jan. 1, 2024 - Jan. 31, 2024
Involved external institutions
United Kingdom (GB)How to cite
APA:
Dombrowski, M., Reynaud, H., Müller, J., Baugh, M., & Kainz, B. (2024). Trade-Offs in Fine-Tuned Diffusion Models between Accuracy and Interpretability. In Association for the Advancement of Artificial Intelligence (Eds.), AAAI-24 Special Track Safe, Robust and Responsible AI Track (pp. 21037-21045). Vancouver, CA: Washington, DC: AAAI Press.
MLA:
Dombrowski, Mischa, et al. "Trade-Offs in Fine-Tuned Diffusion Models between Accuracy and Interpretability." Proceedings of the 38th AAAI Conference on Artificial Intelligence, AAAI 2024, Vancouver Ed. Association for the Advancement of Artificial Intelligence, Washington, DC: AAAI Press, 2024. 21037-21045.
BibTeX: Download