A multimodal approach for automatic generation of radiology reports using chest X-ray images, clinical free-text, and spoken commands.

FAU own research funding: EFI / IZKF / EAM ...


Start date : 15.01.2024

End date : 14.01.2025


Project details

Short description

Advancements in Artificial Intelligence (AI) methods have enabled the development of Large Language Models (LLMs) capable of generating information from user instructions and supporting various tasks in education, research, healthcare, and others. AI has also impacted the field of medical imaging with several deep learning models capable of achieving expert-level performance across different tasks, e.g., detection, segmentation, and assisted clinical diagnosis. In addition, open-source Automatic Speech Recognition (ASR) systems can be incorporated as modules in AI-based systems. This proposed funded project aims to combine LLMs, medical imaging, and speech recognition using AI methods to generate high-quality radiology reports from chest X-ray images.

Scientific Abstract

Advancements in Artificial Intelligence (AI) methods have enabled the development of Large Language Models (LLMs) capable of generating information from user instructions and supporting various tasks in education, research, healthcare, and others. AI has also impacted the field of medical imaging with several deep learning models capable of achieving expert-level performance across different tasks, e.g., detection, segmentation, and assisted clinical diagnosis. In addition, open-source Automatic Speech Recognition (ASR) systems can be incorporated as modules in AI-based systems. This proposed funded project aims to combine LLMs, medical imaging, and speech recognition using AI methods to generate high-quality radiology reports from chest X-ray images.

Involved:

Contributing FAU Organisations:

Research Areas