Title: A topology-aware method for scientific application deployment on cloud

Authors: Pei Fan; Zhenbang Chen; Ji Wang; Zibin Zheng; Michael R. Lyu

Addresses: China HuaYi Broadcasting Corporation, Fuzhou 350001, China ' National Key Laboratory for Parallel and Distributed Processing, School of Computer Science, National University of Defense Technology, Changsha 410073, China ' National Key Laboratory for Parallel and Distributed Processing, School of Computer Science, National University of Defense Technology, Changsha 410073, China ' Shenzhen Research Institute, The Chinese University of Hong Kong, Nanshan District, Shenzhen, China; Department of Computer Science & Engineering, The Chinese University of Hong Kong, Hong Kong, China ' Shenzhen Research Institute, The Chinese University of Hong Kong, Nanshan District, Shenzhen, China; Department of Computer Science & Engineering, The Chinese University of Hong Kong, Hong Kong, China

Abstract: Nowadays, more and more scientific applications are moving to cloud computing. The optimal deployment of scientific applications is critical for providing good services to users. Scientific applications are usually topology-aware applications. Therefore, considering the topology of a scientific application during the development will benefit the performance of the application. However, it is challenging to automatically discover and make use of the communication pattern of a scientific application while deploying the application on cloud. To attack this challenge, in this paper, we propose a framework to discover the communication topology of a scientific application by pre-execution and multi-scale graph clustering, based on which the deployment can be optimised. In addition, we present a set of efficient collective operations for cloud based on the common interconnect topology. Comprehensive experiments are conducted by employing a well-known MPI benchmark and comparing the performance of our method with those of other methods. The experimental results show the effectiveness of our topology-aware deployment method.

Keywords: topology awareness; communication topology; scientific applications; application deployment; cloud computing; graph clustering.

DOI: 10.1504/IJWGS.2014.064937

International Journal of Web and Grid Services, 2014 Vol.10 No.4, pp.338 - 370

Received: 22 Jan 2014
Accepted: 15 Mar 2014
Published online: 29 Oct 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article