Aligning Correlation Information for Domain Adaptation in Action Recognition

Page view(s)
77
Checked on May 29, 2024
Aligning Correlation Information for Domain Adaptation in Action Recognition
Title:
Aligning Correlation Information for Domain Adaptation in Action Recognition
Journal Title:
IEEE Transactions on Neural Networks and Learning Systems
Publication Date:
18 October 2022
Citation:
Xu, Y., Cao, H., Mao, K., Chen, Z., Xie, L., & Yang, J. (2022). Aligning Correlation Information for Domain Adaptation in Action Recognition. IEEE Transactions on Neural Networks and Learning Systems, 1–12. https://doi.org/10.1109/tnnls.2022.3212909
Abstract:
Domain adaptation (DA) approaches address domain shift and enable networks to be applied to different scenarios. Although various image DA approaches have been proposed in recent years, there is limited research toward video DA. This is partly due to the complexity in adapting the different modalities of features in videos, which includes the correlation features extracted as long-range dependencies of pixels across spatiotemporal dimensions. The correlation features are highly associated with action classes and proven their effectiveness in accurate video feature extraction through the supervised action recognition task. Yet correlation features of the same action would differ across domains due to domain shift. Therefore, we propose a novel adversarial correlation adaptation network (ACAN) to align action videos by aligning pixel correlations. ACAN aims to minimize the distribution of correlation information, termed as pixel correlation discrepancy (PCD). Additionally, video DA research is also limited by the lack of cross-domain video datasets with larger domain shifts. We, therefore, introduce a novel HMDB-ARID dataset with a larger domain shift caused by a larger statistical difference between domains. This dataset is built in an effort to leverage current datasets for dark video classification. Empirical results demonstrate the state-of-the-art performance of our proposed ACAN for both existing and the new video DA datasets.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - Career Development Award
Grant Reference no. : C210112046

This research / project is supported by the Nanyang Technological University - Presidential Postdoctoral Fellowship
Grant Reference no. : NA
Description:
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
ISSN:
2162-2388
2162-237X
Files uploaded:

File Size Format Action
acan-tnnls-published-version-amend.pdf 2.26 MB PDF Request a copy