PCA-based Hotelling's T2 chart with fast minimum covariance determinant (FMCD) estimator and kernel density estimation (KDE) for network intrusion detection

Abstract In this work, the combination between the Principal Component Analysis (PCA) and the Hotelling’s T2 chart is proposed to solve problems caused by the many highly correlated network traffic features and to reduce the computational time without reducing its accuracy detection. However, a new issue arises due to the difficulty of the network traffic observations to follow the multivariate normal distribution as required in Hotelling’s T2 chart. Consequently, many false alarms are found in inspecting network intrusion detection. To solve this issue, the Kernel Density Estimation (KDE) procedure is applied to obtain an optimum control limit. Also, to improve the accuracy detection, the Fast Minimum Covariance Determinant (FMCD) is employed to estimate the robust mean vector and covariance matrix. Experiments using the simulated dataset are conducted to assess the proposed chart’s performance in detecting the presence of outlier for the normal and non-normal of multivariate data. According to the simulation studies, the proposed chart yields higher accuracy and a high detection rate with a low false alarm rate. Further, the proposed Intrusion Detection System (IDS) is utilized in scanning attacks. The reputable KDD99 data is used as the benchmark to make a fair comparison between the proposed IDS and some algorithms. The monitoring outputs show that the proposed approach produces advancements in the speed of computational time with 87.42% of time efficiency. Compared to the other charts in detecting intrusion, the proposed chart produces the lower False Negative Rate (FNR). Also, compared to some classifiers the proposed chart yields a higher accuracy at about 0.9893.