J. H. M. Wong, M. Huzaifah, N. F. Chen and A. T. Aw, "Speech in-context learning of paralinguistic tasks," 2025 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Honolulu, HI, USA, 2025, pp. 1-7, doi: 10.1109/ASRU65441.2025.11434787.
Abstract:
In-context learning adapts a large language model to a new task, without computationally expensive parameter updates. This has previously been demonstrated for text tasks, as well as speech tasks that rely primarily on lexical information, such as recognition and translation. This paper proposes to extend this investigation to consider the ability of current open-source models to exhibit in-context learning on speech tasks that require an understanding of paralinguistic information. The tasks of stutter detection, pronunciation assessment, and speech emotion recognition are investigated. The results suggest that current open-source models already exhibit some degree of speech in-context learning on paralinguistic tasks. To more fully utilise available adaptation data, it is also proposed to overcome the finite number of in-context exemplars allowed by the model's prompt length limit, through ensemble combination over multiple in-context learning runs that each use different exemplars.
License type:
Publisher Copyright
Funding Info:
There was no specific funding for the research done