Unsupervised Anomaly Detection and Root Cause Analysis in Mobile Networks

Telecommunication networks are designed for high reliability; however, when they do fail, it is difficult to detect and diagnose problems in a timely manner as the networks are characteristically complex and the problems may be subtle rather than catastrophic. Within the overall objective of network anomaly detection, the focus of this paper is on three aspects: Dimensionality reduction through feature selection to determine the minimum set of features that provide the most information regarding the network state; the application of the multivariate unsupervised learning technique, Principal Component Analysis (PCA), to detect anomalies with low detection latency; and root cause analyses using finite state machines. For anomaly detection, we apply PCA on the normal data, and find the subspace where the variation of the normal data is small. By characterizing the variation of the normal data in the subspace, we derive a boundary of the normal data, and develop an anomaly detection model based on it. Once the anomalies are detected, we compare the message patterns of the anomaly data to those of the normal data to determine where the problems are occurring. Moreover, we examine the error codes in the anomaly data to better understand the underlying problems. The algorithms are developed and tested with Per Call Measurement Data (PCMD) records from a 4G-LTE network. The impact of our work is the proactive detection and root cause analysis of network anomalies in mobile networks, thereby improving network reliability and availability.