Senone-Aware Adversarial Multi-Task Training for Unsupervised Child to Adult Speech Adaptation

Senone-Aware Adversarial Multi-Task Training for Unsupervised Child to Adult Speech Adaptation

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

10.1109/ICASSP39728.2021.9413738

https://doi.org/10.1109/icassp39728.2021.9413738

Richeng Duan, Nancy F. Chen

13 May 2021

Duan, R., & Chen, N. F. (2021). Senone-Aware Adversarial Multi-Task Training for Unsupervised Child to Adult Speech Adaptation. ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). doi:10.1109/icassp39728.2021.9413738

Acoustic modeling for child speech is challenging due to the high acoustic variability caused by physiological differences in the vocal tract. The dearth of publicly available datasets makes the task more challenging. In this work, we propose a feature adaptation approach by exploiting adversarial multi-task training to minimize acoustic mismatch at the senone (tied triphone states) level between adult and child speech and leverage large amounts of transcribed adult speech. We validate the proposed method on three tasks: child speech recognition, child pronunciation assessment and child fluency score prediction. Empirical results indicate that our proposed approach consistently outperforms competitive baselines, achieving 7.7% relative error reduction on speech recognition and up to 25.2% relative gains on the evaluation tasks.

Publisher Copyright

There was no specific funding for the research done

© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

https://oar.a-star.edu.sg/communities-collections/articles/17586

2379-190X
1520-6149

978-1-7281-7605-5
978-1-7281-7606-2

Institute for Infocomm Research