An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series

Page view(s)
30
Checked on Sep 04, 2022
An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series
Title:
An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series
Other Titles:
IEEE Transactions on Neural Networks and Learning Systems
Publication Date:
31 August 2021
Citation:
Garg, A., Zhang, W., Samaran, J., Savitha, R., & Foo, C.-S. (2021). An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series. IEEE Transactions on Neural Networks and Learning Systems, 1–10. doi:10.1109/tnnls.2021.3105827
Abstract:
Several techniques for multivariate time series anomaly detection have been proposed recently, but a systematic comparison on a common set of datasets and metrics is lacking. This article presents a systematic and comprehensive evaluation of unsupervised and semisupervised deep-learning-based methods for anomaly detection and diagnosis on multivariate time series data from cyberphysical systems. Unlike previous works, we vary the model and post-processing of model errors, i.e., the scoring functions independently of each other, through a grid of ten models and four scoring functions, comparing these variants to state-of-the-art methods. In time-series anomaly detection, detecting anomalous events is more important than detecting individual anomalous time points. Through experiments, we find that the existing evaluation metrics either do not take events into account or cannot distinguish between a good detector and trivial detectors, such as a random or an all-positive detector. We propose a new metric to overcome these drawbacks, namely, the composite F-score (Fc_1), for evaluating time-series anomaly detection. Our study highlights that dynamic scoring functions work much better than static ones for multivariate time series anomaly detection, and the choice of scoring functions often matters more than the choice of the underlying model. We also find that a simple, channel-wise model--the univariate fully connected auto-encoder, with the dynamic Gaussian scoring function emerges as a winning candidate for both anomaly detection and diagnosis, beating state-of-the-art algorithms.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - HBMS IAF-PP
Grant Reference no. : H19/01/a0/023

This research / project is supported by the A*STAR - AME Programmatic Funds
Grant Reference no. : A20H6b0151
Description:
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
ISSN:
2162-237X
2162-2388
Files uploaded:

Files uploaded:

File Size Format Action
an-evaluation-of-anomaly-detection-and-diagnosis-in-multivariate-time-series.pdf 2.60 MB PDF Request a copy