Title: Cost-aware hybrid cloud scheduling of parameter sweep calculations using predictive algorithms

Authors: Stig Bosmans; Glenn Maricaux; Filip Van Der Schueren; Peter Hellinckx

Addresses: IMEC, IDLab, Faculty of Applied Engineering, University of Antwerp, Antwerp, Belgium ' Faculty of Applied Engineering, University of Antwerp, Antwerp, Belgium ' Faculty of Applied Engineering, University of Antwerp, Antwerp, Belgium ' IMEC, IDLab, Faculty of Applied Engineering, University of Antwerp, Antwerp, Belgium

Abstract: This paper investigates various techniques for scheduling the costs of parameter sweep calculations efficiently in a hybrid cloud environment. The combination of both a private and a public cloud environment integrates the advantages of being cost effective and having virtually unlimited scaling capabilities at the same time. To make an accurate estimate for the required resources, multiple prediction techniques are discussed. The estimation can be used to create an efficient scheduler which respects both deadline and cost. These findings have been implemented and tested in a Java-based cloud framework which operates on Amazon EC2 and OpenNebula. Also, we present a theoretical model to further optimise the cost by leveraging the Amazon Spot Market.

Keywords: parameter sweep; cloud computing; Amazon AWS EC2; predictive algorithms; OpenNebula; Amazon spot market; machine learning.

DOI: 10.1504/IJGUC.2019.097221

International Journal of Grid and Utility Computing, 2019 Vol.10 No.1, pp.63 - 75

Received: 23 Jan 2017
Accepted: 08 Sep 2017

Published online: 24 Dec 2018 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article