Int. J. of Wireless and Mobile Computing   »   2015 Vol.9, No.2



Title: Prospects, challenges and latest developments in designing a scalable big data stream computing system


Authors: Dawei Sun; Chunxiao Liu; Dongfeng Ren


Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China; School of Information Engineering, China University of Geosciences, Beijing 100083, China
College of Information Science and Technology, Bohai University, Jinzhou, Liaoning 121013, China
Network Security Department, Neusoft Corporation, Shenyang 110179, China


Abstract: In the big data era, big data stream computing, as a computing paradigm, is gaining traction for real-time and online data computing applications, and is specially designed to solve the dilemma of real-time data stream computing by processing data online within real-time constraints. It is used to compute large amounts of data in the form of continuous data streams. Each computation is represented by a data stream graph, usually a directed graph. In this paper, the computing paradigm of big data stream computation in data stream graph is given, and some application scenarios and big data stream characteristics are presented. A series of challenges in designing a scalable big data stream computing system in big data environments are summarised by referring to some research results of big data stream computing system, which include stateless system architecture, elastically adaptive scheduling strategy and fine-grained fault tolerance strategy. All these challenges will greatly help us to understand big data steam computing, and provide a high-level guidance in designing an excellent and useful big data stream computing system.


Keywords: big data stream computing; task scheduling; system design; stateless system architecture; adaptive scheduling; fine-grained fault tolerance.


DOI: 10.1504/IJWMC.2015.072567


Int. J. of Wireless and Mobile Computing, 2015 Vol.9, No.2, pp.155 - 160


Submission date: 13 Apr 2015
Date of acceptance: 06 May 2015
Available online: 19 Oct 2015



Editors Full text accessAccess for SubscribersPurchase this articleComment on this article