Foundation Models in Medical Image Segmentation: How Foundational Are Foundation Models Really?

Lurz D, Neubig L, Kopp M, Kist A (2026)

Publication Type: Conference contribution

Publication year: 2026

Journal

Informatik aktuell Gesellschaft für Informatik (GI)

Publisher: Springer Science and Business Media Deutschland GmbH

Pages Range: 40-47

Conference Proceedings Title: Informatik aktuell

Event location: Lübeck

ISBN: 9783658510992

DOI: 10.1007/978-3-658-51100-5_8

Abstract

Task-constrained deep learning models have shown strong performance for medical image segmentation. Recently, generalist segmentation foundation models have emerged, showing promising results across different scenarios. However, we lack large-scale studies comparing the performance of 2D image, video, and volume segmentation across multiple models and modalities. To unmask how foundational the models truly are, we comprehensively evaluate the segmentation performance of SAM2.1, SAM3, MedSAM2, SAM-Med2D, SAM-Med3D, nnInteractive and VISTA3D on more than 80 medical datasets. MedSAM2stands out as the most foundational of all models, while SAM-Med3D and VISTA3D excel in 3D CT segmentation scenarios, but require noticeably more computational power and memory. nnInteractive seems to be a promising model, featuring very fast inference time and rather high segmentation performance. Our code and evaluation results are openly available at https://github.com/DavidL-11/med-seg-fm.

Authors with CRIS profile

Luisa Neubig Juniorprofessur für Artificial Intelligence in Communication Disorders Markus Kopp Institute of Radiology Andreas Kist Juniorprofessur für Artificial Intelligence in Communication Disorders

How to cite

APA:

Lurz, D., Neubig, L., Kopp, M., & Kist, A. (2026). Foundation Models in Medical Image Segmentation: How Foundational Are Foundation Models Really? In Heinz Handels, Katharina Breininger, Thomas Deserno, Andreas Maier, Klaus Maier-Hein, Christoph Palm, Thomas Tolxdorff (Eds.), Informatik aktuell (pp. 40-47). Lübeck, DE: Springer Science and Business Media Deutschland GmbH.

MLA:

Lurz, David, et al. "Foundation Models in Medical Image Segmentation: How Foundational Are Foundation Models Really?" Proceedings of the Bildverarbeitung für die Medizin Workshop, BVM 2026, Lübeck Ed. Heinz Handels, Katharina Breininger, Thomas Deserno, Andreas Maier, Klaus Maier-Hein, Christoph Palm, Thomas Tolxdorff, Springer Science and Business Media Deutschland GmbH, 2026. 40-47.

BibTeX: Download