Class-incremental Learning for Time Series: Benchmark and Evaluation

Class-incremental Learning for Time Series: Benchmark and Evaluation

Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

10.1145/3637528.3671581

http://dx.doi.org/10.1145/3637528.3671581

Zhongzheng Qiao, Quang Pham, Zhen Cao, Hoang H. Le, P. N. Suganthan, Xudong Jiang, Savitha Ramasamy

class incremental learning, Continual Learning, time-series data analysis, benchmark evaluation

25 August 2024

Qiao, Z., Pham, Q., Cao, Z., Le, H. H., Suganthan, P. N., Jiang, X., & Ramasamy, S. (2024). Class-incremental Learning for Time Series: Benchmark and Evaluation. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 33, 5613–5624. https://doi.org/10.1145/3637528.3671581

Real-world environments are inherently non-stationary, frequently introducing new classes over time. This is especially common in time series classification, such as the emergence of new disease classification in healthcare or the addition of new activities in human activity recognition. In such cases, a learning system is required to assimilate novel classes effectively while avoiding catastrophic forgetting of the old ones, which gives rise to the Class-incremental Learning (CIL) problem. However, despite the encouraging progress in the image and language domains, CIL for time series data remains relatively understudied. Existing studies suffer from inconsistent experimental designs, necessitating a comprehensive evaluation and benchmarking of methods across a wide range of datasets. To this end, we first present an overview of the Time Series Class-incremental Learning (TSCIL) problem, highlight its unique challenges, and cover the advanced methodologies. Further, based on standardized settings, we develop a unified experimental framework that supports the rapid development of new algorithms, easy integration of new datasets, and standardization of the evaluation process. Using this framework, we conduct a comprehensive evaluation of various generic and time-series-specific CIL methods in both standard and privacy-sensitive scenarios. Our extensive experiments not only provide a standard baseline to support future research but also shed light on the impact of various design factors such as normalization layers or memory budget thresholds.

Attribution 4.0 International (CC BY 4.0)

This research / project is supported by the National Research Foundation, Singapore - CNRS@CREATE Programme
Grant Reference no. : NA

https://oar.a-star.edu.sg/communities-collections/articles/20959

979-8-4007-0490-1

Institute for Infocomm Research