Mixed Reality Support for Orthopaedic Surgery - Lee, Sing Chun; Fuerst, Bernhard; Fotouhi, Javad; Johnson, Alex; Tateno, Keisuke; Tombari, Federico; Osgood, Greg; Navab, Nassir (Adjunct Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017, 2017)
This work presents a mixed reality environment for orthopaedic interventions that provides a 3D overlay of Cone-beam CT images, surgical site, and real-time tool tracking. The system uses an RGBD camera attached to the detector plane of a mobile C-arm, which is a typical device to acquire X-Ray images during surgery. Calibration of the two devices is done by acquiring simultaneous CBCT and RGBD scans of a calibration phantom and computing the rigid transformation between them. The markerless tracking of the surgical tool is computed in the RGBD view using real-time segmentation and Simultaneous Localization And Mapping. The RGBD view is then overlaid to the CBCT data with real-time point clouds of the surgical site. This visualization provides multiple desired views of the medical data, surgical site, and the tracking of surgical tools, which could be used to provide intuitive visualization for orthopedic procedures to place instrumentation and to assist surgeons with their localization and coordination. Our proposed opto-X-ray system can lead to x-ray radiation dose reduction as well as improved safety in minimally invasive orthopaedic procedures.
CRIS-Team Scopus Importer:2023-03-08
Keywords: Cone-Beam CT; Intra-operative; Mixed Reality; Multi-modal Visualization; Range Imaging; Tracking