Int. J. of Wireless and Mobile Computing   »   2015 Vol.9, No.2

 

 

Title: Prospects, challenges and latest developments in designing a scalable big data stream computing system

 

Authors: Dawei Sun; Chunxiao Liu; Dongfeng Ren

 

Addresses:
Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China; School of Information Engineering, China University of Geosciences, Beijing 100083, China
College of Information Science and Technology, Bohai University, Jinzhou, Liaoning 121013, China
Network Security Department, Neusoft Corporation, Shenyang 110179, China

 

Abstract: In the big data era, big data stream computing, as a computing paradigm, is gaining traction for real-time and online data computing applications, and is specially designed to solve the dilemma of real-time data stream computing by processing data online within real-time constraints. It is used to compute large amounts of data in the form of continuous data streams. Each computation is represented by a data stream graph, usually a directed graph. In this paper, the computing paradigm of big data stream computation in data stream graph is given, and some application scenarios and big data stream characteristics are presented. A series of challenges in designing a scalable big data stream computing system in big data environments are summarised by referring to some research results of big data stream computing system, which include stateless system architecture, elastically adaptive scheduling strategy and fine-grained fault tolerance strategy. All these challenges will greatly help us to understand big data steam computing, and provide a high-level guidance in designing an excellent and useful big data stream computing system.

 

Keywords: big data stream computing; task scheduling; system design; stateless system architecture; adaptive scheduling; fine-grained fault tolerance.

 

DOI: 10.1504/IJWMC.2015.072567

 

Int. J. of Wireless and Mobile Computing, 2015 Vol.9, No.2, pp.155 - 160

 

Submission date: 13 Apr 2015
Date of acceptance: 06 May 2015
Available online: 19 Oct 2015

 

 

Editors Full text accessAccess for SubscribersPurchase this articleComment on this article