Evaluating Speech–Phoneme Alignment and its Impact on Neural Text-To-Speech Synthesis

Zalkow F, Govalkar P, Müller M, Habets E, Dittmar C (2023)

Publication Type: Conference contribution

Publication year: 2023

Publisher: IEEE

Conference Proceedings Title: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Event location: Rhodos GR

DOI: 10.1109/ICASSP49357.2023.10097248

Abstract

In recent years, the quality of text-to-speech (TTS) synthesis vastly improved due to deep-learning techniques, with parallel architectures, in particular, providing excellent synthesis quality at fast inference. Training these models usually requires speech recordings, corresponding phoneme-level transcripts, and the temporal alignment of each phoneme to the utterances. Since manually creating such fine-grained alignments requires expert knowledge and is time-consuming, it is common practice to estimate them using automatic speech–phoneme alignment methods. In the literature, either the estimation methods’ accuracy or their impact on the TTS system’s synthesis quality is evaluated. In this study, we perform experiments with five state-of-the-art speech–phoneme aligners and evaluate their output with objective and subjective measures. As our main result, we show that small alignment errors (below 75 ms error) do not decrease the synthesis quality, which implies that the alignment error may not be the crucial factor when choosing an aligner for TTS training.

Authors with CRIS profile

Meinard Müller International Audio Laboratories Erlangen (AudioLabs) Emanuël Habets Lehrstuhl für wahrnehmungsbasierte räumliche Audiosignalverarbeitung (Stiftungslehrstuhl)

Involved external institutions

Fraunhofer-Institut für Integrierte Schaltungen (IIS) DE Germany (DE)

How to cite

APA:

Zalkow, F., Govalkar, P., Müller, M., Habets, E., & Dittmar, C. (2023). Evaluating Speech–Phoneme Alignment and its Impact on Neural Text-To-Speech Synthesis. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Rhodos, GR: IEEE.

MLA:

Zalkow, Frank, et al. "Evaluating Speech–Phoneme Alignment and its Impact on Neural Text-To-Speech Synthesis." Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodos IEEE, 2023.

BibTeX: Download