Gao, H., Wu, M., Chen, Z., Li, Y., Wang, X., An, S., Li, J., & Liu, C. (2023). SSA-ICL: Multi-domain adaptive attention with intra-dataset continual learning for Facial expression recognition. Neural Networks, 158, 228–238. https://doi.org/10.1016/j.neunet.2022.11.025
Abstract:
Facial expression recognition (FER) is a kind of affective computing that identifies the emotional state represented in facial photographs. Various methods have been developed for completing this critical task. In spite of this progress, three significant obstacles, the interaction between spatial action units, the inadequacy of semantic information about spectral expressions and the unbalanced data distribution, are not well addressed. In this work, we propose SSA-ICL, a novel approach for FER, and solve these three difficulties inside a coherent framework. To address the first two challenges, we develop a Spectral and Spatial Attention (SSA) module that integrates spectral semantics with spatial locations to improve the performance of the model. We provide an Intra-dataset Continual Learning (ICL) module to combat the issue of long-tail distribution in FER datasets. By subdividing a single long-tail dataset into multiple sub-datasets, ICL repeatedly trains well-balanced representations from each subset and finally develop a independent classifier. We performed extensive experiments on two publicly available datasets, AffectNet and RAFDB. In comparison to existing attention modules, our SSA achieves an accuracy improvement of 3.8% ~ 6.7%, as evidenced by testing results. In the meanwhile, our proposed SSA-ICL can achieve superior or comparable performance to state-of-the-art FER methods (65.78% on AffectNet and 89.44% on RAFDB).
License type:
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Funding Info:
This research / project is supported by the National Research Foundation - AI Singapore Programme
Grant Reference no. : AISG2-RP-2021-027
This research was funded by the National Key Research and Development Program of China (2019YFE0113800), the National Natural Science Foundation of China (62171123, 62071241 and 81871444), the Natural Science Foundation of Jiangsu Province (BK20190014 and BK20192004), the Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX20 0088), and the Fundamental Research Funds for the Central Universities (3207032101D).