Web Attack Detection through Network-Traffic-Based Feature Engineering and Machine Learning

The design of feature set is a basic research direction of anomaly detection technology. The characterization ability of feature set directly affects the accuracy and generalization ability of intrusion detection algorithm. KDD CUP 99 is the most widely used data set in anomaly detection related research papers, but the feature set needs to analyze the payload content characteristics of packets, so the feature set cannot be applied to the encrypted traffic. CICIDS2017 is also a widely used data set. However, because data in PCAP format can only be used as input, the generalization ability of the model trained by this data set is poor when it is used for real-time attack detection.Based on the above problems, a feature set based on network flow characteristics is designed and implemented in this paper, which includes the time-window-based statistical features related to network traffic and host-related statistical features. Experiments show that the anomaly detection model trained by the feature set can meet the requirements of both real-time and classification accuracy.