Int. J. of Big Data Intelligence   »   2017 Vol.4, No.1



Title: Optimising column family for OLAP queries in HBase


Authors: Fangzhou Yang; Dragan Milosevic; Jian Cao


Department of CES, Shanghai Jiaotong University, Shanghai, China
ZANOX AG, Stralauer Allee 2, Berlin, Germany
Department of CES, Shanghai Jiaotong University, Shanghai, China


Abstract: Apache HBase is a column-oriented NoSQL key-value store built on top of the Hadoop distributed file-system. Logically, columns in HBase are grouped into column families. Physically, all columns in one column family are stored in the same set of files. Therefore the division of column families is closely related to the response time for a specific row query. In this paper, a new evolutionary algorithm is designed and applied to group columns and find the optimum column family schema for the given user queries. The reading performance of the optimised column family schema is evaluated on a real dataset provided by ZANOX. It is shown that by using the found optimised column family schema, the reading performance while executing OLAP queries is improved with a statistical significance. Queries from a test set show that the average response time is reduced by up to 72% compared to reference column family schemas.


Keywords: HBase; NoSQL; column families; column layout; column grouping; evolutionary algorithms; MapReduce; multi-chromosome; schema optimisation; OLAP queries; Hadoop.


DOI: 10.1504/IJBDI.2016.10002002


Int. J. of Big Data Intelligence, 2017 Vol.4, No.1, pp.23 - 35


Submission date: 01 Jul 2015
Date of acceptance: 26 Jan 2016
Available online: 26 Dec 2016



Editors Full text accessPurchase this articleComment on this article