Title: Community detection in dynamic networks with spark

Authors: Priyangika R. Piyasinghe; J. Morris Chang

Addresses: Department of Computer Science, Iowa State University, Ames, IA, 50011, USA ' Department of Electrical Engineering, University of South Florida, Tampa, FL, 33620, USA

Abstract: Detecting the evolution of communities within dynamically changing networks is important to understand the latent structure of complex large graphs. In this paper, we present an algorithm to detect real-time communities in dynamically changing networks. We demonstrate the proposed methodology through a case study in peer-to-peer (P2P) botnet detection which is one of the major threats to network security for serving as the infrastructure that is responsible for various cyber crimes. Our method considers online community structure from time to time and improves efficiency by maintaining the same level of accuracy of community detection over time. Experimental evaluation on Apache Spark implementation of the method showed that the execution time improves over dynamic version of Girvan-Newman community detection algorithm while having a higher accuracy level.

Keywords: dynamic networks; community detection; Girvan-Newman algorithm; large graphs; spark.

DOI: 10.1504/IJDS.2018.094505

International Journal of Data Science, 2018 Vol.3 No.3, pp.236 - 254

Received: 05 Dec 2016
Accepted: 29 Mar 2017

Published online: 04 Sep 2018 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article