Int. J. of Wireless and Mobile Computing   »   2016 Vol.10, No.4

 

 

Title: Analysing and evaluating topology structure of online application in Big Data stream computing environment

 

Authors: Rui Huang; Dawei Sun

 

Addresses:
School of Information Engineering, China University of Geosciences, Beijing 100083, China
School of Information Engineering, China University of Geosciences, Beijing 100083, China

 

Abstract: Stream computing systems process high-rate incoming data sources in dynamic environments and generate real-time results. A critical problem in stream processing is the difficulty in devising an optimal strategy for configuring topology structure. That is due to the inability to predict or adapt to the dynamics of the data flow intensity and the computing platform's workload. Therefore, the best strategy should rely on the real-time statistics of performance and workload to achieve self-adaptation. In this paper, we first analyse the impact of application topology structures in Storm on performance and resource utilisation efficiency. From the metrics of stream processing application and environment, we derive some effective optimisation rules towards designing an online reconfiguration strategy. Our evaluation of the experimental results reveals the intricacies in searching the configuration parameters space and shows that a strategy based on our rules can substantially improve performance as well as achieve high resource utilisation efficiency.

 

Keywords: big data; stream computing; parallelism adjustment; topology structure; Storm; online applications; data flow intensity; resource utilisation; optimisation rules; online reconfiguration strategy; efficiency; performance evaluation.

 

DOI: 10.1504/IJWMC.2016.078204

 

Int. J. of Wireless and Mobile Computing, 2016 Vol.10, No.4, pp.317 - 324

 

Submission date: 15 Mar 2016
Date of acceptance: 04 Apr 2016
Available online: 08 Aug 2016

 

 

Editors Full text accessAccess for SubscribersPurchase this articleComment on this article