Title: A parallel multi-objective swarm intelligence framework for Big Data analysis

Authors: Amr Mohamed AbdelAziz; Kareem Kamal A. Ghany; Taysir Hassan A. Soliman; Adel Abu El-Magd Sewisy

Addresses: Faculty of Computers and Artificial Intelligence, Beni-Suef University, Beni-Suef 62111, Egypt ' Faculty of Computers and Artificial Intelligence, Beni-Suef University, Beni-Suef 62111, Egypt; College of Computing and Informatics, Saudi Electronic University, Riyadh 11673, Saudi Arabia ' Faculty of Computers and Information, Assiut University, Assiut 71516, Egypt ' Faculty of Computers and Information, Assiut University, Assiut 71516, Egypt

Abstract: Nowadays, data are generated from smart devices in huge volumes, different formats, and high pace, which comply with Big Data characteristics. Big Data led to the emergence of new technologies, such as Hadoop and Spark to provide both data management and analysis. Analysing Big Data is a time-consuming process. Particle swarm and ant colony optimisation are population-based meta-heuristic methods. They have been combined with data mining techniques to solve MultiObjective Problems (MOPs) of small and medium sized data, presenting good performance. However, when applying these methods to solve MOPs in Big data, an efficient scalable framework will be required. In this paper, we summarise new technologies proposed to manage and analyse Big Data. We present how meta-heuristics can be adapted with Big Data technologies. We characterise problems arose when analysing MO Big Data problems, in addition to proposed methods to overcome these problems, giving examples in Bioinformatics field.

Keywords: Big Data; Big Data analysis; data mining; particle swarm optimisation; multi-objective optimisation; MapReduce; Spark.

DOI: 10.1504/IJCAT.2020.109342

International Journal of Computer Applications in Technology, 2020 Vol.63 No.3, pp.200 - 212

Accepted: 14 Mar 2020
Published online: 03 Sep 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article