Title: Pipel: exploiting resource reorganisation to optimise performance of pipeline-structured applications in the cloud

Authors: Vinicius Meyer; Vinicius Facco Rodrigues; Rodrigo Da Rosa Righi; Cristiano André Da Costa; Guilherme Galante; Cristiano Bonato Both

Addresses: Applied Computing Graduate Program, Unisinos University, Av. Unisinos, 950 – Cristo Rei, São Leopoldo, Rio Grande do Sul, Brazil ' Applied Computing Graduate Program, Unisinos University, Av. Unisinos, 950 – Cristo Rei, São Leopoldo, Rio Grande do Sul, Brazil ' Applied Computing Graduate Program, Unisinos University, Av. Unisinos, 950 – Cristo Rei, São Leopoldo, Rio Grande do Sul, Brazil ' Applied Computing Graduate Program, Unisinos University, Av. Unisinos, 950 – Cristo Rei, São Leopoldo, Rio Grande do Sul, Brazil ' Western Paraná State University – UNIOESTE, R. Universitária, 2069 – Jardim Universitário, Cascavel, Paraná, Brazil ' Federal University of Health Sciences of Porto Alegre – UFCSPA, R. Sarmento Leite, 245 – Centro Histórico, Porto Alegre, Rio Grande do Sul, Brazil

Abstract: Workflow has become a standard for many scientific applications that are characterized by a collection of processing elements. Particularly, a pipeline application is a type of workflow that receives a set of tasks, which must pass through all processing elements in a linear fashion. However, the strategy of using a fixed number of resources can cause under- or over-provisioning situations, besides not fitting irregular demands. In this context, our idea is to deploy the pipeline application in the cloud, so executing it with a feature that differentiates cloud from other distributed systems: resource elasticity. Thus, we propose Pipel: a reactive elasticity model that uses lower and upper load thresholds and the CPU metric to on-the-fly select the most appropriated number of compute nodes for each stage along the pipeline execution. The results were promising, presenting an average gain of 38% in the application time when comparing non-elastic and elastic executions.

Keywords: cloud elasticity; pipeline applications; performance optimisation; dynamic resource management; adaptivity.

DOI: 10.1504/IJCSYSE.2019.098414

International Journal of Computational Systems Engineering, 2019 Vol.5 No.1, pp.1 - 17

Received: 02 Jun 2017
Accepted: 03 Aug 2017
Published online: 22 Mar 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article