Zhou, Y., Tian, X., & Li, H. (2020). Multi-Task WaveRNN With an Integrated Architecture for Cross-Lingual Voice Conversion. IEEE Signal Processing Letters, 27, 1310–1314. https://doi.org/10.1109/lsp.2020.3010163
Abstract:
Spoken languages are similar phonetically because humans have a common vocal production system. However, each language has a unique phonetic repertoire and phonotactic rule. In cross-lingual voice conversion, source speaker and target speaker speak different languages. The challenge is how to project the speaker identity of the source speaker to that of the target across two different phonetic systems. A typical voice conversion system employs a generator-vocoder pipeline, where the generator is responsible for conversion, and the vocoder is for waveform reconstruction. We propose a novel Multi-Task WaveRNN with an integrated architecture for cross-lingual voice conversion. The WaveRNN is trained on two sets of monolingual data via a two-task learning. The integrated architecture takes linguistic features as input and outputs speech waveform directly. Voice conversion experiments are conducted between English and Mandarin, which confirm the effectiveness of the proposed method in terms of speech quality and speaker similarity.
License type:
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Funding Info:
This research / project is supported by the Agency for Science, Technology and Research (A*STAR) - AME Programmatic Funding Scheme
Grant Reference no. : A18A2b0046
This research / project is supported by the National Research Foundation (NRF) Singapore - Human-Robot Interaction Phase 1, National Robotics Programme
Grant Reference no. : 192 25 00054
This research / project is supported by the National Research Foundation (NRF) Singapore - AI Speech Lab, AI Singapore Programme
Grant Reference no. : AISG-100E-2018-006