Exploring GPT-4 as MR Sequence and Reconstruction Programming Assistant

Zaiß M, Rajput JR, Dang HN, Golkov V, Cremers D, Knoll F, Maier A (2024)

Publication Language: English

Publication Type: Conference contribution, Conference Contribution

Publication year: 2024

Journal

Informatik aktuell Gesellschaft für Informatik (GI)

Publisher: Springer Vieweg

Series: Informatik aktuell

City/Town: Wiesbaden

Pages Range: 94-99

Conference Proceedings Title: Bildverarbeitung für die Medizin 2024. BVM 2024

Event location: Erlangen DE

ISBN: 9783658440367

DOI: 10.1007/978-3-658-44037-4_28

Abstract

In this study, we explore the potential of generative pre-trained transformer (GPT), as a coding assistant for MRI sequence programming using the Pulseq framework. The programming of MRI sequences is traditionally a complex and time-consuming task, and the Pulseq standard has recently simplified this process. It allows researchers to define and generate complex pulse sequences used in MRI experiments. Leveraging GPT-4’s capabilities in natural language generation, we adapted it for MRI sequence programming, creating a specialized assistant named GPT4MR. Our tests involved generating various MRI sequences, revealing that GPT-4, guided by a tailored prompt, outperformed GPT-3.5, producing fewer errors and demonstrating improved reasoning. Despite limitations in handling complex sequences, GPT4MR corrected its own errors and successfully generated code with step-by-step instructions. The study showcases GPT4MR’s ability to accelerate MRI sequence development, even for novel ideas absent in its training set. While further research and improvement are needed to address complexity limitations, a well-designed prompt enhances performance. The findings propose GPT4MR as a valuable MRI sequence programming assistant, streamlining prototyping and development. The future prospect involves integrating a PyPulseq plugin into lightweight, open-source LLMs, potentially revolutionizing MRI sequence development and prototyping.

Authors with CRIS profile

Moritz Zaiß Professur für Multimodale Bildgebung in der klinischen Forschung Hoai Nam Dang Neuroradiologisches Institut Florian Knoll Professur für Computational Imaging Andreas Maier Lehrstuhl für Informatik 5 (Mustererkennung)

Involved external institutions

Technische Universität München (TUM) DE Germany (DE)

How to cite

APA:

Zaiß, M., Rajput, J.R., Dang, H.N., Golkov, V., Cremers, D., Knoll, F., & Maier, A. (2024). Exploring GPT-4 as MR Sequence and Reconstruction Programming Assistant. In Andreas Maier, Thomas M. Deserno, Heinz Handels, Klaus Maier-Hein, Christoph Palm, Thomas Tolxdorff (Eds.), Bildverarbeitung für die Medizin 2024. BVM 2024 (pp. 94-99). Erlangen, DE: Wiesbaden: Springer Vieweg.

MLA:

Zaiß, Moritz, et al. "Exploring GPT-4 as MR Sequence and Reconstruction Programming Assistant." Proceedings of the German Conference on Medical Image Computing, BVM 2024, Erlangen Ed. Andreas Maier, Thomas M. Deserno, Heinz Handels, Klaus Maier-Hein, Christoph Palm, Thomas Tolxdorff, Wiesbaden: Springer Vieweg, 2024. 94-99.

BibTeX: Download