Domain-agnostic weakly supervised surgical instrument segmentation

Peter R, Pham DXV, Matten P, Wu E, Nienhaus J, Meissen F, Menten MJ, Tagliabue E, Mathis-Ullrich F (2026)

Publication Type: Journal article

Publication year: 2026

Journal

Scientific Reports Nature Publishing Group: Open Access Journals - Option B

Book Volume: 16

Article Number: 9337

Journal Issue: 1

DOI: 10.1038/s41598-026-43054-1

Abstract

Recent advancements in visual foundation models open new avenues in the field of surgical instrument segmentation in medical images. Segmentation foundation models provide high segmentation accuracy for objects of interest that are selected via prompts in the form of points, bounding boxes, or text. However, the choice of suitable prompts either requires manual interaction or relies on two-stage pipelines based on supervised, typically domain-specific models. This limits their applicability for domain-agnostic surgical instrument segmentation. We propose a method for surgical instrument segmentation that leverages the power of the segmentation foundation model SAM2 while eliminating the need for a user-defined input prompt or domain-specific annotated datasets. We achieve this by utilizing an anomaly detector generated from non-instrument images to identify instruments as unseen regions and in this way, define a SAM2 input prompt based solely on image-level annotations. For three datasets for surgical instrument segmentation from diverse domains (EndoVis2017, CaDIS, and PASO-SIS), we achieve mean Normalized Surface Distances ranging from. This demonstrates the competitiveness of our method compared to alternatives, while its training- and mask-free nature makes it well-suited for surgical workflow integration. By simplifying surgical instrument segmentation, we advance the field of computer-assisted surgery and unlock a wide variety of assistance functions with minimal effort.

Authors with CRIS profile

Franziska Mathis-Ullrich Professur für Robotische Planung und Kognition in der Chirurgie

Involved external institutions

Carl Zeiss AG DE Germany (DE) Medizinische Universität Wien AT Austria (AT) Technische Universität München (TUM) DE Germany (DE)

How to cite

APA:

Peter, R., Pham, D.X.V., Matten, P., Wu, E., Nienhaus, J., Meissen, F.,... Mathis-Ullrich, F. (2026). Domain-agnostic weakly supervised surgical instrument segmentation. Scientific Reports, 16(1). https://doi.org/10.1038/s41598-026-43054-1

MLA:

Peter, Rebekka, et al. "Domain-agnostic weakly supervised surgical instrument segmentation." Scientific Reports 16.1 (2026).

BibTeX: Download