On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansion

On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansion

2023 IEEE/CVF International Conference on Computer Vision (ICCV)

10.1109/ICCV51070.2023.01087

http://dx.doi.org/10.1109/iccv51070.2023.01087

Yushu Li, Xun Xu, Yongyi Su, Kui Jia

test-time training, out of distribution, open-world test-time training

15 January 2024

Li, Y., Xu, X., Su, Y., & Jia, K. (2023, October 1). On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansion. 2023 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv51070.2023.01087

Generalizing deep learning models to unknown target domain distribution with low latency has motivated research into test-time training/adaptation (TTT/TTA). Existing approaches often focus on improving test-time training performance under well-curated target domain data. As figured out in this work, many state-of-the-art methods fail to maintain the performance when the target domain is contaminated with strong out-of-distribution (OOD) data, a.k.a. open-world test-time training (OWTTT). The failure is mainly due to the inability to distinguish strong OOD samples from regular weak OOD samples. To improve the robustness of OWTTT we first develop an adaptive strong OOD pruning which improves the efficacy of the self-training TTT method. We further propose a way to dynamically expand the prototypes to represent strong OOD samples for an improved weak/strong OOD data separation. Finally, we regularize self-training with distribution alignment and the combination yields the state-of-the-art performance on 5 OWTTT benchmarks. The code is available at https://github.com/Yushu-Li/OWTTT.

Publisher Copyright

This research / project is supported by the A*STAR - MTC Programmatic Fund
Grant Reference no. : M23L7b0021

This research / project is supported by the A*STAR - Project
Grant Reference no. : C210112059

This research / project is supported by the National Natural Science Foundation of China - Grant
Grant Reference no. : 62106078

This research / project is supported by the Sichuan Science and Technology Program - Project
Grant Reference no. : 2023NSFSC1421

This research / project is supported by the Program for Guangdong Introducing Innovative and Entrepreneurial Teams - Project
Grant Reference no. : 2017ZT07X183

This research / project is supported by the Guangdong R&D key project of China - Project
Grant Reference no. : 2019B010155001

© 2023 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

https://oar.a-star.edu.sg/communities-collections/articles/20627

2380-7504

Institute for Infocomm Research

https://openaccess.thecvf.com/content/ICCV2023/papers/Li_On_the_Robustness_of_Open-World_Test-Time_Training_Self-Training_with_Dynamic_ICCV_2023_paper.pdf