A multi-sensor architecture combining human pose estimation and real-time location systems for workflow monitoring on hybrid operating suites

Rodrigues VF, Antunes RS, Seewald LA, Bazo R, dos Reis ES, dos Santos UJ, Righi RdR, da S. LG, da Costa CA, Bertollo FL, Maier A, Eskofier B, Horz T, Pfister M, Fahrig R (2022)

Publication Type: Journal article

Publication year: 2022

Journal

Future Generation Computer Systems-The International Journal of Grid Computing Theory Methods and Applications Elsevier

Book Volume: 135

Pages Range: 283-298

DOI: 10.1016/j.future.2022.05.006

Abstract

Despite the advancements to improve patient safety, a significant number of errors still occur in Operating Suites (OS). To improve medical decision-making and the resulting quality of care, it is essential to monitor and understand medical activities’ workflow and interactions. Although some strategies employ different sensor devices, their focus is not on generating complete workflow information. They only combine different data sources to generate their final output, lacking at least one piece of information from a workflow. To tackle this challenge, this paper presents [Formula presented], a distributed architecture model for sensor data acquisition and processing. [Formula presented] ’s main contribution lies in its multi-sensor data fusion algorithms to extract a computational representation of activities in surgical procedures. In addition, [Formula presented] is flexible to accommodate different deployment configurations,combining depth cameras, Ultra-Wideband positioning systems,and deep learning-based human pose estimation (HPE) mechanisms. The workflow monitoring mechanism was deployed in an actual hybrid OS for an extensive evaluation of the proposal. Our experiments demonstrate that the architecture can capture the information required to monitor the surgical workflow. In particular, the proposed HPE methodology accurately detects the poses of medical staff members, with a maximum error of 5cm.

Authors with CRIS profile

Andreas Maier Lehrstuhl für Informatik 5 (Mustererkennung) Björn Eskofier Machine Learning and Data Analytics Lab

Involved external institutions

Siemens AG DE Germany (DE) Unisinos (Universidade do Vale do Rio dos Sinos) BR Brazil (BR) University Foundation of Cardiology / Fundação Universitária de Cardiologia (FUC) BR Brazil (BR)

How to cite

APA:

Rodrigues, V.F., Antunes, R.S., Seewald, L.A., Bazo, R., dos Reis, E.S., dos Santos, U.J.,... Fahrig, R. (2022). A multi-sensor architecture combining human pose estimation and real-time location systems for workflow monitoring on hybrid operating suites. Future Generation Computer Systems-The International Journal of Grid Computing Theory Methods and Applications, 135, 283-298. https://dx.doi.org/10.1016/j.future.2022.05.006

MLA:

Rodrigues, Vinicius F., et al. "A multi-sensor architecture combining human pose estimation and real-time location systems for workflow monitoring on hybrid operating suites." Future Generation Computer Systems-The International Journal of Grid Computing Theory Methods and Applications 135 (2022): 283-298.

BibTeX: Download