Title: Availability modelling and assurance for a big data computing

Authors: Zuqiang Ke; Nohpill Park

Addresses: Computer Science Department, Oklahoma State University, Stillwater, OK, USA ' Computer Science Department, Oklahoma State University, Stillwater, OK, USA

Abstract: This paper proposes a new analytical model to evaluate the availability of a big data computing, namely, map-reduce computing on a Hadoop platform. Map-reduce computing is represented by a queueing model in this work in order to trace flow of tasks (either map or reduce) of their arrivals and exits in the course of computation. The objective of the model is to evaluate the probability for a map-reduce computation to be available at an instance of time, referred to as availability. The set of variables taken into account in this model lists the number of map and reduce tasks, the number of servers (or referred to as nodes in this paper) engaged, along with a few constants such as task arrival/exit rates and node failure/repair rates. The proposed model provides a comprehensive yet fundamental basis to assure and ultimately optimise the design of map-reduce computing in terms of availability with reference to its performance in a simultaneous manner. Parametric simulations have been conducted and demonstrated the efficacy of the proposed model in assessing the availability and the cost.

Keywords: availability; map-reduce computing; queueing model.

DOI: 10.1504/IJBDI.2019.098915

International Journal of Big Data Intelligence, 2019 Vol.6 No.2, pp.113 - 128

Received: 09 Mar 2018
Accepted: 26 Sep 2018
Published online: 09 Apr 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article