Video-Based Activity Recognition for Automated Motor Assessment of Parkinson's Disease

Sarapata G, Dushin Y, Morinan G, Ong J, Budhdeo S, Kainz B, O'Keeffe J (2023)

Publication Type: Journal article

Publication year: 2023

Journal

IEEE Journal of Biomedical and Health Informatics Institute of Electrical and Electronics Engineers (IEEE)

Book Volume: 27

Pages Range: 5032-5041

Journal Issue: 10

DOI: 10.1109/JBHI.2023.3298530

Abstract

Over the last decade, video-enabled mobile devices have become ubiquitous, while advances in markerless pose estimation allow an individual's body position to be tracked accurately and efficiently across the frames of a video. Previous work by this and other groups has shown that pose-extracted kinematic features can be used to reliably measure motor impairment in Parkinson's disease (PD). This presents the prospect of developing an asynchronous and scalable, video-based assessment of motor dysfunction. Crucial to this endeavour is the ability to automatically recognise the class of an action being performed, without which manual labelling is required. Representing the evolution of body joint locations as a spatio-temporal graph, we implement a deep-learning model for video and frame-level classification of activities performed according to part 3 of the Movement Disorder Society Unified PD Rating Scale (MDS-UPDRS). We train and validate this system using a dataset of n = 7310 video clips, recorded at 5 independent sites. This approach reaches human-level performance in detecting and classifying periods of activity within monocular video clips. Our framework could support clinical workflows and patient care at scale through applications such as quality monitoring of clinical data collection, automated labelling of video streams, or a module within a remote self-assessment system.

Authors with CRIS profile

Bernhard Kainz

Involved external institutions

Machine Medicine Technologies GB United Kingdom (GB) University College London (UCL) GB United Kingdom (GB) Imperial College London / The Imperial College of Science, Technology and Medicine GB United Kingdom (GB)

How to cite

APA:

Sarapata, G., Dushin, Y., Morinan, G., Ong, J., Budhdeo, S., Kainz, B., & O'Keeffe, J. (2023). Video-Based Activity Recognition for Automated Motor Assessment of Parkinson's Disease. IEEE Journal of Biomedical and Health Informatics, 27(10), 5032-5041. https://dx.doi.org/10.1109/JBHI.2023.3298530

MLA:

Sarapata, Grzegorz, et al. "Video-Based Activity Recognition for Automated Motor Assessment of Parkinson's Disease." IEEE Journal of Biomedical and Health Informatics 27.10 (2023): 5032-5041.

BibTeX: Download