Title: Live data migration approach from relational tables to schema-free collections with MapReduce

Authors: Kun Ma; Fusen Dong

Addresses: Shandong Provincial Key Laboratory of Network-Based Intelligent Computing, University of Jinan, Jinan 250022, Shandong, China ' School of Information Science and Engineering, University of Jinan, Jinan 250022, Shandong, China

Abstract: Although NoSQL has some new features to address the query bottleneck of big data, the hybrid solution of relational database and NoSQL will last for a long period. Therefore, recent researches focus on the data migration issue from relational database to document stores. However, there are few publications on live data migration in parallel. In this paper, we attempt to address this old problem using new MapReduce framework. The process of migration consists of log-based change data capture, merging of changed data, and blocking and transformation. We utilise predicate logic of mathematical relation and QVT relations to describe the mapping rules clearly. Finally, our experimental evaluation of log-based blocking and transformation with MapReduce shows the higher effectiveness and efficiency than current methods.

Keywords: live data migration; NoSQL; relational tables; MapReduce; big data; schema-free collections; relational databases; change data capture; changed data merging; predicate logic; mapping rules; log-based blocking; log-based transformation.

DOI: 10.1504/IJSTM.2015.073942

International Journal of Services Technology and Management, 2015 Vol.21 No.4/5/6, pp.318 - 335

Received: 02 Jan 2015
Accepted: 30 Jun 2015

Published online: 30 Dec 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article