Spark Based Intrusion Detection System Using Practical Swarm Optimization Clustering

Given the availability growth of data in large networks, intrusion detection systems become an important challenge since they require efficient methods to discover attacks from such networks. This paper proposes a new Spark based intrusion detection system using particle swarm optimization clustering, referred to as IDS-SPSO, for large scale data able to provide good tradeoff between scalability and accuracy. The use of Particle swarm optimization clustering is argued to avoid the sensitivity problem of initial cluster centers as well as premature convergence. In addition, we propose in this work to take advantage of parallel processing based on the Spark framework. Experiments performed on several large collections of real intrusion data have shown the effectiveness of the proposed intrusion detection system in terms of scalability and clustering accuracy.