Yu, J., Yang, M., Yan, J., & Deng, C. (2025). CLIP-based autonomous visual prompting for unsupervised domain incremental learning. Neurocomputing, 652, 131019. https://doi.org/10.1016/j.neucom.2025.131019
Abstract:
Domain Incremental Learning (DIL) focuses on updating the model with knowledge from novel domains (i.e., currently being processed domain) while maintaining the knowledge acquired from previous domains, mitigating the catastrophic forgetting in deep learning. In real-world applications, annotating a large-scale dataset for DIL is expensive and time-consuming. This paper focuses on a more realistic and challenging task: Open-Vocabulary Unsupervised Domain Incremental Learning (OV-UDIL) assumes that all the data in the novel domain are unlabeled during the incremental learning stage, and the vocabulary list is opened across all domains during training. Considering that Vision-Language Models (VLMs) contain abundant semantic knowledge, they can be applied to many downstream tasks in a zero-shot setting. Therefore, we propose a novel CLIP-based Autonomous Visual Prompting (AVP) approach for UDIL. Specifically, we first generate and tune the visual prompt using the unlabeled data from the current domain to uncover and leverage the knowledge embedded in CLIP. Then, we leverage visual prompts from the previous domain to adjust the corresponding prompts in the current domain, thereby preserving each prompt’s particular attribute. Overall, AVP alleviates the adverse effects of domain feature shifts on CLIP’s generalization capability, enhancing recognition performance on the UDIL task. Extensive experiments on four popular datasets demonstrate that our method achieves superior performance compared with state-of-the-art DIL approaches.
License type:
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Funding Info:
There was no specific funding for the research done