The use of phonetic motor invariants can improve automatic phoneme discrimination

Castellini C, Badino L, Metta G, Sandini G, Tavella M, Grimaldi M, Fadiga L (2011)

Publication Type: Journal article

Publication year: 2011

Journal

PLoS ONE Public Library of Science

Book Volume: 6

Article Number: e24055

Journal Issue: 9

DOI: 10.1371/journal.pone.0024055

Abstract

We investigate the use of phonetic motor invariants (MIs), that is, recurring kinematic patterns of the human phonetic articulators, to improve automatic phoneme discrimination. Using a multi-subject database of synchronized speech and lips/tongue trajectories, we first identify MIs commonly associated with bilabial and dental consonants, and use them to simultaneously segment speech and motor signals. We then build a simple neural network-based regression schema (called Audio-Motor Map, AMM) mapping audio features of these segments to the corresponding MIs. Extensive experimental results show that (a) a small set of features extracted from the MIs, as originally gathered from articulatory sensors, are dramatically more effective than a large, state-of-the-art set of audio features, in automatically discriminating bilabials from dentals; (b) the same features, extracted from AMM-reconstructed MIs, are as effective as or better than the audio features, when testing across speakers and coarticulating phonemes; and dramatically better as noise is added to the speech signal. These results seem to support some of the claims of the motor theory of speech perception and add experimental evidence of the actual usefulness of MIs in the more general framework of automated speech recognition. © 2011 Castellini et al.

Authors with CRIS profile

Claudio Castellini

Involved external institutions

University of Genova / Università degli Studi di Genova IT Italy (IT) Italian Institute of Technology / Istituto Italiano di Tecnologia (IIT) IT Italy (IT) Università del Salento IT Italy (IT)

How to cite

APA:

Castellini, C., Badino, L., Metta, G., Sandini, G., Tavella, M., Grimaldi, M., & Fadiga, L. (2011). The use of phonetic motor invariants can improve automatic phoneme discrimination. PLoS ONE, 6(9). https://dx.doi.org/10.1371/journal.pone.0024055

MLA:

Castellini, Claudio, et al. "The use of phonetic motor invariants can improve automatic phoneme discrimination." PLoS ONE 6.9 (2011).

BibTeX: Download