On the Synthesis of Dysarthric Speech: Evaluating Zero-Shot and Fine-Tuning Approaches

Matoušek J, Šmídl L, Perez Toro PA, Nöth E, Lehečka J, Maier A, Arias Vergara T (2026)


Publication Type: Conference contribution

Publication year: 2026

Journal

Publisher: Springer Science and Business Media Deutschland GmbH

Book Volume: 16940 LNCS

Pages Range: 300-314

Conference Proceedings Title: Lecture Notes in Computer Science

Event location: Brno CZ

ISBN: 9783032372482

DOI: 10.1007/978-3-032-37249-9_25

Abstract

The critical scarcity of pathological speech data significantly hinders the development of robust automatic speech recognition (ASR) systems for individuals with speech impairments. To address this issue, this paper investigates the capabilities of modern neural text-to-speech (TTS) architectures to synthesize highly accurate Spanish dysarthric speech. We explore two distinct synthesis strategies (zero-shot voice cloning and model fine-tuning) and two TTS models (ZipVoice and StyleTTS2) applied to recordings of 50 speakers with Parkinson’s Disease and 50 healthy controls. To evaluate the quality and preservation of speaker identity in the generated samples, we employ WavLM-based speaker embeddings alongside detailed acoustic-phonetic analyses. By systematically comparing real and synthesized speech, we aim to determine the optimal TTS training approaches for capturing dysarthric characteristics, thereby establishing a crucial foundation for future ASR data augmentation efforts. Experimental results demonstrate that model fine-tuning significantly outperforms zero-shot approaches, with the fine-tuned StyleTTS2 emerging as the most effective configuration, achieving the highest speaker similarity and demonstrating superior robustness across classification tasks and phonetic/prosodic analyses.

Authors with CRIS profile

Involved external institutions

How to cite

APA:

Matoušek, J., Šmídl, L., Perez Toro, P.A., Nöth, E., Lehečka, J., Maier, A., & Arias Vergara, T. (2026). On the Synthesis of Dysarthric Speech: Evaluating Zero-Shot and Fine-Tuning Approaches. In Elmar Nöth, Aleš Horák, Petr Sojka (Eds.), Lecture Notes in Computer Science (pp. 300-314). Brno, CZ: Springer Science and Business Media Deutschland GmbH.

MLA:

Matoušek, Jindřich, et al. "On the Synthesis of Dysarthric Speech: Evaluating Zero-Shot and Fine-Tuning Approaches." Proceedings of the 29th International Conference on Text, Speech, and Dialogue, TSD 2026, Brno Ed. Elmar Nöth, Aleš Horák, Petr Sojka, Springer Science and Business Media Deutschland GmbH, 2026. 300-314.

BibTeX: Download