An Automatic Multimodal Approach to Analyze Linguistic and Acoustic Cues on Parkinson's Disease Patients
Escobar-Grisales D, Arias-Vergara T, Rios-Urrego CD, Nöth E, García AM, Orozco-Arroyave JR (2023)
Publication Language: English
Publication Type: Conference contribution
Publication year: 2023
Publisher: International Speech Communication Association
Book Volume: 2023-August
Pages Range: 1703-1707
Conference Proceedings Title: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
Event location: Dublin, IRL
DOI: 10.21437/Interspeech.2023-2287
Abstract
Early detection and monitoring of Parkinson's disease are crucial for properly treating and managing the symptoms. Automatic speech and language analysis has emerged as a promising non-invasive method to monitor the patient's state. This study analyzed different speech and language representations for automatic classification between Parkinson's disease patients and healthy controls. First, each modality is analyzed independently. General representations such as Wav2vec or BETO are used together with representations oriented to model disease traits such as phonemic identifiability in speech modality and grammatical units analysis in language modality. The best speech and language representations were combined using a fusion strategy based on Gated Multimodal Units. The best results are achieved with the multimodal approach, outperforming all results obtained with unimodal representations and the traditional fusion strategy.
Authors with CRIS profile
Involved external institutions
Universidad de Antioquía (UDEA)
Colombia (CO)
University of California San Francisco (UCSF)
United States (USA) (US)
Colombia (CO)
University of California San Francisco (UCSF)
United States (USA) (US)
How to cite
APA:
Escobar-Grisales, D., Arias-Vergara, T., Rios-Urrego, C.D., Nöth, E., García, A.M., & Orozco-Arroyave, J.R. (2023). An Automatic Multimodal Approach to Analyze Linguistic and Acoustic Cues on Parkinson's Disease Patients. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (pp. 1703-1707). Dublin, IRL, IE: International Speech Communication Association.
MLA:
Escobar-Grisales, Daniel, et al. "An Automatic Multimodal Approach to Analyze Linguistic and Acoustic Cues on Parkinson's Disease Patients." Proceedings of the 24th International Speech Communication Association, Interspeech 2023, Dublin, IRL International Speech Communication Association, 2023. 1703-1707.
BibTeX: Download