Generalizing Reinforcement Learning through Fusing Self-Supervised Learning into Intrinsic Motivation

Page view(s)
17
Checked on Mar 25, 2024
Generalizing Reinforcement Learning through Fusing Self-Supervised Learning into Intrinsic Motivation
Title:
Generalizing Reinforcement Learning through Fusing Self-Supervised Learning into Intrinsic Motivation
Journal Title:
36th AAAI Conference on Artificial Intelligence
DOI:
Keywords:
Publication Date:
22 February 2022
Citation:
K. Wu, M. Wu, Z. Chen, Y. Xu, X. Li, Generalizing Reinforcement Learning through Fusing Self-Supervised Learning into Intrinsic Motivation, AAAI, 2022.
Abstract:
Despite the great potential of reinforcement learning (RL) in solving complex decision-making problems, generalization remains one of its key challenges, leading to difficulty in deploying learned RL policies to new environments. In this paper, we propose to improve the generalization of RL algorithms through fusing Self-supervised learning into Intrinsic Motivation (SIM). Specifically, SIM boosts representation learning through driving the cross-correlation matrix between the embeddings of augmented and non-augmented samples close to the identity matrix. This aims to increase the similarity between the embedding vectors of a sample and its augmented version while minimizing the redundancy between the components of these vectors. Meanwhile, the redundancy reduction based self-supervised loss is converted to an intrinsic reward to further improve generalization in RL via an auxiliary objective. As a general paradigm, SIM can be implemented on top of any RL algorithm. Extensive evaluations have been performed on a diversity of tasks. Experimental results demonstrate that SIM consistently outperforms the state-of-the-art methods and exhibits superior generalization capability and sample efficiency.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - Career Development Award
Grant Reference no. : C210112046
Description:
ISSN:
TBC
Files uploaded:

File Size Format Action
7240-wuk.pdf 2.96 MB PDF Open