An Automatic Multimodal Approach to Analyze Linguistic and Acoustic Cues on Parkinson's Disease Patients

Escobar-Grisales D, Arias-Vergara T, Rios-Urrego CD, Nöth E, García AM, Orozco-Arroyave JR (2023)


Publication Language: English

Publication Type: Conference contribution

Publication year: 2023

Publisher: International Speech Communication Association

Book Volume: 2023-August

Pages Range: 1703-1707

Conference Proceedings Title: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Event location: Dublin, IRL IE

DOI: 10.21437/Interspeech.2023-2287

Abstract

Early detection and monitoring of Parkinson's disease are crucial for properly treating and managing the symptoms. Automatic speech and language analysis has emerged as a promising non-invasive method to monitor the patient's state. This study analyzed different speech and language representations for automatic classification between Parkinson's disease patients and healthy controls. First, each modality is analyzed independently. General representations such as Wav2vec or BETO are used together with representations oriented to model disease traits such as phonemic identifiability in speech modality and grammatical units analysis in language modality. The best speech and language representations were combined using a fusion strategy based on Gated Multimodal Units. The best results are achieved with the multimodal approach, outperforming all results obtained with unimodal representations and the traditional fusion strategy.

Authors with CRIS profile

Involved external institutions

How to cite

APA:

Escobar-Grisales, D., Arias-Vergara, T., Rios-Urrego, C.D., Nöth, E., García, A.M., & Orozco-Arroyave, J.R. (2023). An Automatic Multimodal Approach to Analyze Linguistic and Acoustic Cues on Parkinson's Disease Patients. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (pp. 1703-1707). Dublin, IRL, IE: International Speech Communication Association.

MLA:

Escobar-Grisales, Daniel, et al. "An Automatic Multimodal Approach to Analyze Linguistic and Acoustic Cues on Parkinson's Disease Patients." Proceedings of the 24th International Speech Communication Association, Interspeech 2023, Dublin, IRL International Speech Communication Association, 2023. 1703-1707.

BibTeX: Download