Title: Research on parallelisation of collaborative filtering recommendation algorithm based on Spark

Authors: Yongli Yang; Zhenhu Ning; Yongquan Cai; Peng Liang; Haifeng Liu

Addresses: Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China ' Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China ' Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China ' State of Information Center, National Engineering Laboratory for E-government Integration and Application, Beijing 100045, China ' Science and Technology on Information Systems, Engineering Laboratory, Beijing Institute of Control and Electronic Technology, Beijing 100038, China

Abstract: More and more people become conscious of the recommendation system to make good use of the data through their inherent advantages faced with the large amount of data on the Internet. The collaborative filtering recommendation algorithm cannot avoid the bottleneck of computing performance problems in the recommendation process. In this paper, we propose a parallel collaborative filtering recommendation algorithm RLPSO_KM_CF which is implemented based on Spark. Firstly, the RLPSO (reverse-learning and local-learning PSO) algorithm is used to find the optimal solution of particle swarm and output the optimised clustering centre. Then, the RLPSO_KM algorithm is used to cluster the user information. Finally, make effective recommendations to the target user by combining the traditional user-based collaborative filtering algorithm with the RLPSO_KM clustering algorithm. The experimental results show that the RLPSO_KM_CF algorithm has a significant improvement in the recommendation accuracy and has a higher speed-up and stability.

Keywords: collaborative filtering recommendation algorithm; RLPSO algorithm; K-means algorithm; Spark.

DOI: 10.1504/IJWMC.2018.093856

International Journal of Wireless and Mobile Computing, 2018 Vol.14 No.4, pp.312 - 319

Received: 15 Sep 2017
Accepted: 01 Feb 2018
Published online: 07 Aug 2018 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article