Multitask Weakly Supervised Generative Network for MR-US Registration

Azampour MF, Mach K, Fatemizadeh E, Demiray B, Westenfelder K, Steiger K, Eiber M, Wendler T, Kainz B, Navab N (2024)

Publication Type: Journal article

Publication year: 2024

Journal

IEEE Transactions on Medical Imaging Institute of Electrical and Electronics Engineers (IEEE)

Pages Range: 1-1

DOI: 10.1109/TMI.2024.3400899

Abstract

Registering pre-operative modalities, such as magnetic resonance imaging or computed tomography, to ultrasound images is crucial for guiding clinicians during surgeries and biopsies. Recently, deep-learning approaches have been proposed to increase the speed and accuracy of this registration problem. However, all of these approaches need expensive supervision from the ultrasound domain. In this work, we propose a multitask generative framework that needs weak supervision only from the pre-operative imaging domain during training. To perform a deformable registration, the proposed framework translates a magnetic resonance image to the ultrasound domain while preserving the structural content. To demonstrate the efficacy of the proposed method, we tackle the registration problem of pre-operative 3D MR to transrectal ultrasonography images as necessary for targeted prostate biopsies. We use an in-house dataset of 600 patients, divided into 540 for training, 30 for validation, and the remaining for testing. An expert manually segmented the prostate in both modalities for validation and test sets to assess the performance of our framework. The proposed framework achieves a 3.58 mm target registration error on the expert-selected landmarks, 89.2% in the Dice score, and 1.81 mm 95th percentile Hausdorff distance on the prostate masks in the test set. Our experiments demonstrate that the proposed generative model successfully translates magnetic resonance images into the ultrasound domain. The translated image contains the structural content and fine details due to an ultrasound-specific two-path design of the generative model. The proposed framework enables training learning-based registration methods while only weak supervision from the pre-operative domain is available.

Authors with CRIS profile

Bernhard Kainz Professur für Image Data Exploration and Analysis

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Klinikum rechts der Isar DE Germany (DE) Johns Hopkins University (JHU) US United States (USA) (US) Sharif University of Technology / دانشگاه صنعتی شریف (SUT) IR Iran, Islamic Republic of (IR)

How to cite

APA:

Azampour, M.F., Mach, K., Fatemizadeh, E., Demiray, B., Westenfelder, K., Steiger, K.,... Navab, N. (2024). Multitask Weakly Supervised Generative Network for MR-US Registration. IEEE Transactions on Medical Imaging, 1-1. https://doi.org/10.1109/TMI.2024.3400899

MLA:

Azampour, Mohammad Farid, et al. "Multitask Weakly Supervised Generative Network for MR-US Registration." IEEE Transactions on Medical Imaging (2024): 1-1.

BibTeX: Download