Title: Design and application of real-time network abnormal traffic detection system based on Spark Streaming
Authors: FuCheng Pan; DeZhi Han; Yuping Hu
Addresses: College of Information Engineering, Shanghai Maritime University, Shanghai, 201306,China ' College of Information Engineering, Shanghai Maritime University, Shanghai, 201306,China ' School of Information, Guangdong University of Finance and Economics, Guangdong, 510320, China
Abstract: In order to realise the rapid analysis and identification of abnormal traffic in real-time networks, a distributed real-time network abnormal traffic detection system (DRNATDS) was designed, which could effectively analyse abnormal network traffic. DRNATDS provided effective real-time big data analysis platform and guaranteed network security. The paper proposes K-means algorithm based on relative density and distance, integrated with Spark Streaming and Kafka. It could effectively detect various network attacks under real-time data stream. The experimental results show that DRNATDS has good high availability and stability. Compared to other algorithms, K-means algorithm based on relative density and distance could more effectively identify abnormal network traffic and improve the recognition rate.
Keywords: Spark Streaming; Kafka; network abnormal traffic identification; K-means.
International Journal of Embedded Systems, 2019 Vol.11 No.5, pp.562 - 572
Available online: 20 Sep 2019 *Full-text access for editors Access for subscribers Purchase this article Comment on this article