Title: Optimising column family for OLAP queries in HBase

Authors: Fangzhou Yang; Dragan Milosevic; Jian Cao

Addresses: Department of CES, Shanghai Jiaotong University, Shanghai, China ' ZANOX AG, Stralauer Allee 2, Berlin, Germany ' Department of CES, Shanghai Jiaotong University, Shanghai, China

Abstract: Apache HBase is a column-oriented NoSQL key-value store built on top of the Hadoop distributed file-system. Logically, columns in HBase are grouped into column families. Physically, all columns in one column family are stored in the same set of files. Therefore the division of column families is closely related to the response time for a specific row query. In this paper, a new evolutionary algorithm is designed and applied to group columns and find the optimum column family schema for the given user queries. The reading performance of the optimised column family schema is evaluated on a real dataset provided by ZANOX. It is shown that by using the found optimised column family schema, the reading performance while executing OLAP queries is improved with a statistical significance. Queries from a test set show that the average response time is reduced by up to 72% compared to reference column family schemas.

Keywords: HBase; NoSQL; column families; column layout; column grouping; evolutionary algorithms; MapReduce; multi-chromosome; schema optimisation; OLAP queries; Hadoop.

DOI: 10.1504/IJBDI.2017.081195

International Journal of Big Data Intelligence, 2017 Vol.4 No.1, pp.23 - 35

Received: 04 Jul 2015
Accepted: 26 Jan 2016

Published online: 26 Dec 2016 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article