Home Publications Research Grants Inventions & Patents Awards Additional Research Activities Faculties & Institutions Research Areas

STTAR: Surgical Tool Tracking using Off-the-Shelf Augmented Reality Head-Mounted Displays

Martin-Gomez A, Li H, Song T, Yang S, Wang G, Ding H, Navab N, Zhao Z, Armand M (2023)

---

Publication Type: Journal article

Publication year: 2023

Journal

Pages Range: 1-16

DOI: 10.1109/TVCG.2023.3238309

Abstract

The use of Augmented Reality (AR) for navigation purposes has shown beneficial in assisting physicians during the performance of surgical procedures. These applications commonly require knowing the pose of surgical tools and patients to provide visual information that surgeons can use during the performance of the task. Existing medical-grade tracking systems use infrared cameras placed inside the Operating Room (OR) to identify retro-reflective markers attached to objects of interest and compute their pose. Some commercially available AR Head-Mounted Displays (HMDs) use similar cameras for self-localization, hand tracking, and estimating the objects' depth. This work presents a framework that uses the built-in cameras of AR HMDs to enable accurate tracking of retro-reflective markers without the need to integrate any additional electronics into the HMD. The proposed framework can simultaneously track multiple tools without having previous knowledge of their geometry and only requires establishing a local network between the headset and a workstation. Our results show that the tracking and detection of the markers can be achieved with an accuracy of $0.09\pm 0.06\ mm$ on lateral translation, $0.42 \pm 0.32\ mm$ on longitudinal translation and $0.80 \pm 0.39^\circ$ for rotations around the vertical axis. Furthermore, to showcase the relevance of the proposed framework, we evaluate the system's performance in the context of surgical procedures. This use case was designed to replicate the scenarios of k-wire insertions in orthopedic procedures. For evaluation, seven surgeons were provided with visual navigation and asked to perform 24 injections using the proposed framework. A second study with ten participants served to investigate the capabilities of the framework in the context of more general scenarios. Results from these studies provided comparable accuracy to those reported in the literature for AR-based navigation procedures.

Involved external institutions

How to cite

APA:

Martin-Gomez, A., Li, H., Song, T., Yang, S., Wang, G., Ding, H.,... Armand, M. (2023). STTAR: Surgical Tool Tracking using Off-the-Shelf Augmented Reality Head-Mounted Displays. IEEE Transactions on Visualization and Computer Graphics, 1-16. https://dx.doi.org/10.1109/TVCG.2023.3238309

MLA:

Martin-Gomez, Alejandro, et al. "STTAR: Surgical Tool Tracking using Off-the-Shelf Augmented Reality Head-Mounted Displays." IEEE Transactions on Visualization and Computer Graphics (2023): 1-16.

BibTeX: Download