Gao, X., Zhang, H., & Chen, N. F. (2025). Prompt-Unseen-Emotion: Mixed Emotional Speech Synthesis With Prompt-LLM Contextual Knowledge. IEEE Signal Processing Letters, 32, 4259–4263. https://doi.org/10.1109/lsp.2025.3627104
Abstract:
Existing expressive text-to-speech (TTS) systems primarily model a limited set of categorical emotions, whereas human conversations extend far beyond these predefined emotions, making it essential to explore more diverse emotional speech generation for more natural interactions. To bridge this gap, this paper proposes a novel prompt-unseen-emotion (PUE) approach to generate unseen emotional speech via emotion-guided prompt learning. PUE is trained utilizing an LLM-TTS architecture to ensure emotional consistency between categorical emotion-relevant prompts and emotional speech, allowing the model to quantitatively capture different emotion weightings per utterance. During inference, mixed emotional speech can be generated by flexibly adjusting emotion proportions and leveraging LLM contextual knowledge, enabling the model to quantify different emotional styles. Our proposed PUE successfully facilitates expressive speech synthesis of unseen emotions in a zero-shot setting.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the National Research Foundation - AI Singapore - National Large Language Models Funding Initiative
Grant Reference no. : AISG-NMLP-2024-004
This research / project is supported by the A∗STAR - Japan-Singapore Joint Call: Japan Science and Technology Agency (JST)
Grant Reference no. : R24I6IR136
This research / project is supported by the National Research Foundation - Campus for Research Excellence and Technological Enterprise (CREATE) programme (DesCartes)
Grant Reference no. :