Title: Provisioning big data applications as services on containerised cloud: a microservices-based approach

Authors: Jing Gao; Wubin Li; Zhuofeng Zhao; Yanbo Han

Addresses: Cloud Computing Research Center, North China University of Technology, Beijing, China ' Ericsson Research, Montréal, Canada ' Cloud Computing Research Center, North China University of Technology, Beijing, China ' Cloud Computing Research Center, North China University of Technology, Beijing, China

Abstract: We present the fundamental challenges for dynamic provisioning of big data applications. The findings are based on our previous experience in this domain, as well as a comprehensive study on a selected set of state-of-the-art tools in the big data ecosystem. We then incorporate these findings in a framework aiming at dynamically provisioning big data applications as services on containerised cloud. The innovations behind the framework are to optimise the whole lifecycle of big data applications in a holistic manner by the adoption of microservices(μServices) methodologies. The feasibility of our approach is verified through a case study of provisioning a large-scale user traffic data processing application in a private cloud environment backed by Kubernetes. We also show that while hosting big data applications in containerised cloud can significantly eliminate the presumed complexity of deployment and operation, it in the meantime also comes with a certain amount of cost in terms of learning curve and traceability. Our research helps technical decision makers to assess the adoption of microservices for big data applications more objectively.

Keywords: big data application; cloud computing; container; microservices; automation.

DOI: 10.1504/IJSTM.2020.106744

International Journal of Services Technology and Management, 2020 Vol.26 No.2/3, pp.167 - 181

Received: 10 Jul 2018
Accepted: 26 Mar 2019

Published online: 20 Apr 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article