Point Cloud Segmentation for Autonomous Clip Positioning in Laparoscopic Cholecystectomy on a Phantom

Gyenes B, Franke N, Scheikl P, Henrich P, Younis R, Neumann G, Wagner M, Mathis-Ullrich F (2025)

Publication Type: Journal article

Publication year: 2025

Journal

IEEE Robotics and Automation Letters IEEE

Article Number: 0b00006494228ca6

DOI: 10.1109/LRA.2025.3585357

Abstract

High-risk applications in robotics, such as robotassisted surgery, present unique challenges. These systems must be both highly precise and interpretable in order to be deployed in environments with very low tolerance for error or unsafe exploration. We present the first robotic system to demonstrate autonomous clip positioning on a physical phantom in laparoscopic surgery, one of the most common interventions in general surgery. After segmentation of a colorless point cloud from a single camera, target positions for the clips are extracted using spline interpolation, and can then be adjusted by the human operator. The segmentation model is trained on only 60 handlabeled real point clouds, reflecting data scarcity in the surgical domain. We overcome this with a combination of pre-training on 128,000 synthetic point clouds and two novel data augmentation techniques. The motion of the end-effector to each target is visualized for the operator, satisfying the unique motion constraints of minimally-invasive surgery while ensuring that the robot’s actions are verifiable and interpretable. In real robot experiments, our system localizes targets with the required precision of 0.75 mm at a 95% success rate and executes autonomous clip positioning with a 100% success rate. We provide insights that are applicable to many other surgical and non-surgical tasks that require identifying and navigating to a precise target.

Authors with CRIS profile

Paul Scheikl Professur für Robotische Planung und Kognition in der Chirurgie Pit Henrich Professur für Robotische Planung und Kognition in der Chirurgie Franziska Mathis-Ullrich Professur für Robotische Planung und Kognition in der Chirurgie

Involved external institutions

Karlsruhe Institute of Technology (KIT) DE Germany (DE) Universitätsklinikum Carl Gustav Carus Dresden DE Germany (DE)

How to cite

APA:

Gyenes, B., Franke, N., Scheikl, P., Henrich, P., Younis, R., Neumann, G.,... Mathis-Ullrich, F. (2025). Point Cloud Segmentation for Autonomous Clip Positioning in Laparoscopic Cholecystectomy on a Phantom. IEEE Robotics and Automation Letters. https://doi.org/10.1109/LRA.2025.3585357

MLA:

Gyenes, Balazs, et al. "Point Cloud Segmentation for Autonomous Clip Positioning in Laparoscopic Cholecystectomy on a Phantom." IEEE Robotics and Automation Letters (2025).

BibTeX: Download