Anomaly Detection and Attribution in Networks with Temporally Correlated Traffic

Anomaly Detection and Attribution in Networks with Temporally Correlated Traffic
Anomaly Detection and Attribution in Networks with Temporally Correlated Traffic
Other Titles:
IEEE/ACM Transactions on Networking
Publication Date:
08 December 2017
I. Nevat et al., "Anomaly Detection and Attribution in Networks With Temporally Correlated Traffic," in IEEE/ACM Transactions on Networking, vol. 26, no. 1, pp. 131-144, Feb. 2018. doi: 10.1109/TNET.2017.2765719
Anomaly detection in communication networks is the first step in the challenging task of securing a network, as anomalies may indicate suspicious behaviors, attacks, network malfunctions or failures. In this work, we address the problem of not only detecting the anomalous events, but also of attributing the anomaly to the flows causing it. To this end, we develop a new statistical decision theoretic framework for temporally correlated traffic in networks via Markov Chain modelling. We first formulate the optimal anomaly detection problem via the Generalized Likelihood Ratio Test (GLRT) for our composite model. This results in a combinatorial optimization problem which is prohibitively expensive. We then develop two low-complexity anomaly detection algorithms. The first is based on the Cross Entropy (CE) method, which detects anomalies as well as attributes anomalies to flows. The second algorithm performs anomaly detection via GLRT on the aggregated flows transformation — a compact low dimensional representation of the raw traffic flows. The two algorithms complement each other and allow the network operator to first activate the flow aggregation algorithm in order to quickly detect anomalies in the system. Once an anomaly has been detected, the operator can further investigate which specific flows are anomalous by running the CE based algorithm. We perform extensive performance evaluations, and experiment our algorithms on synthetic and semi-synthetic data, as well as on real Internet traffic data obtained from the MAWI archive, and finally make recommendations regarding their usability.
License type:
Funding Info:
This material is based on research work supported by the Singapore National Research Foundation under NCR Award No. NRF2014NCR-NCR001-034.
© 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Files uploaded: