K. Wu, M. Wu, Z. Chen, Y. Xu, X. Li, Generalizing Reinforcement Learning through Fusing Self-Supervised Learning into Intrinsic Motivation, AAAI, 2022.
Abstract:
Despite the great potential of reinforcement learning (RL)
in solving complex decision-making problems, generalization remains one of its key challenges, leading to difficulty
in deploying learned RL policies to new environments. In
this paper, we propose to improve the generalization of RL
algorithms through fusing Self-supervised learning into Intrinsic Motivation (SIM). Specifically, SIM boosts representation learning through driving the cross-correlation matrix
between the embeddings of augmented and non-augmented
samples close to the identity matrix. This aims to increase the
similarity between the embedding vectors of a sample and
its augmented version while minimizing the redundancy between the components of these vectors. Meanwhile, the redundancy reduction based self-supervised loss is converted
to an intrinsic reward to further improve generalization in RL
via an auxiliary objective. As a general paradigm, SIM can be
implemented on top of any RL algorithm. Extensive evaluations have been performed on a diversity of tasks. Experimental results demonstrate that SIM consistently outperforms the
state-of-the-art methods and exhibits superior generalization
capability and sample efficiency.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - Career Development Award
Grant Reference no. : C210112046