Multimodal Continuous Emotion Recognition: A Technical Report for ABAW5

Page view(s)
42
Checked on Nov 20, 2024
Multimodal Continuous Emotion Recognition: A Technical Report for ABAW5
Title:
Multimodal Continuous Emotion Recognition: A Technical Report for ABAW5
Journal Title:
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Keywords:
Publication Date:
14 August 2023
Citation:
Zhang, S., Zhao, Z., & Guan, C. (2023). Multimodal Continuous Emotion Recognition: A Technical Report for ABAW5. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). https://doi.org/10.1109/cvprw59228.2023.00611
Abstract:
We used two multimodal models for continuous valence-arousal recognition using visual, audio, and linguistic information. The first model is the same as we used in ABAW2 and ABAW3, which employs the leader-follower attention. The second model has the same architecture for spatial and temporal encoding. As for the fusion block, it employs a compact and straightforward channel attention, borrowed from the End2You toolkit. Unlike our previous attempts that use Vggish feature directly as the audio feature, this time we feed the pre-trained VGG model using logmel-spectrogram and finetune it during the training. To make full use of the data and alleviate over-fitting, cross-validation is carried out. The code is available at https://github.com/sucv/ABAW3.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the ASTAR - RIE2020 AME Programmatic Fund
Grant Reference no. : A20G8b0102
Description:
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
ISSN:
2160-7516
Files uploaded:

File Size Format Action
230310335pdf-safe-updated.pdf 1.64 MB PDF Request a copy