Preclinical evaluation of a markerless, real-time, augmented reality guidance system for robot-assisted radical prostatectomy

Kalia M, Avinash A, Navab N, Salcudean S (2021)

Publication Type: Journal article

Publication year: 2021

Journal

International Journal of Computer Assisted Radiology and Surgery Springer Verlag (Germany)

Book Volume: 16

Pages Range: 1181-1188

Journal Issue: 7

DOI: 10.1007/s11548-021-02419-9

Abstract

Purpose: Intra-operative augmented reality (AR) during surgery can mitigate incomplete cancer removal by overlaying the anatomical boundaries extracted from medical imaging data onto the camera image. In this paper, we present the first such completely markerless AR guidance system for robot-assisted laparoscopic radical prostatectomy (RALRP) that transforms medical data from transrectal ultrasound (TRUS) to endoscope camera image. Moreover, we reduce the total number of transformations by combining the hand–eye and camera calibrations in a single step. Methods: Our proposed solution requires two transformations: TRUS to robot, DVTTRUS, and camera projection matrix, M (i.e., the transformation from endoscope to camera image frame). DVTTRUS is estimated by the method proposed in Mohareri et al. (in J Urol 193(1):302–312, 2015). M is estimated by selecting corresponding 3D-2D data points in the endoscope and the image coordinate frame, respectively, by using a CAD model of the surgical instrument and a preoperative camera intrinsic matrix with an assumption of a projective camera. The parameters are estimated using Levenberg–Marquardt algorithm. Overall mean re-projection errors (MRE) are reported using simulated and real data using a water bath. We show that M can be re-estimated if the focus is changed during surgery. Results: Using simulated data, we received an overall MRE in the range of 11.69–13.32 pixels for monoscopic and stereo left and right cameras. For the water bath experiment, the overall MRE is in the range of 26.04–30.59 pixels for monoscopic and stereo cameras. The overall system error from TRUS to camera world frame is 4.05 mm. Details of the procedure are given in supplementary material. Conclusion: We demonstrate a markerless AR guidance system for RALRP that does not need calibration markers and thus has the capability to re-estimate the camera projection matrix if it changes during surgery, e.g., due to a focus change.

Involved external institutions

University of British Columbia CA Canada (CA) Technische Universität München (TUM) DE Germany (DE)

How to cite

APA:

Kalia, M., Avinash, A., Navab, N., & Salcudean, S. (2021). Preclinical evaluation of a markerless, real-time, augmented reality guidance system for robot-assisted radical prostatectomy. International Journal of Computer Assisted Radiology and Surgery, 16(7), 1181-1188. https://doi.org/10.1007/s11548-021-02419-9

MLA:

Kalia, Megha, et al. "Preclinical evaluation of a markerless, real-time, augmented reality guidance system for robot-assisted radical prostatectomy." International Journal of Computer Assisted Radiology and Surgery 16.7 (2021): 1181-1188.

BibTeX: Download