Title: Optimising column family for OLAP queries in HBase
Authors: Fangzhou Yang; Dragan Milosevic; Jian Cao
Addresses: Department of CES, Shanghai Jiaotong University, Shanghai, China ' ZANOX AG, Stralauer Allee 2, Berlin, Germany ' Department of CES, Shanghai Jiaotong University, Shanghai, China
Abstract: Apache HBase is a column-oriented NoSQL key-value store built on top of the Hadoop distributed file-system. Logically, columns in HBase are grouped into column families. Physically, all columns in one column family are stored in the same set of files. Therefore the division of column families is closely related to the response time for a specific row query. In this paper, a new evolutionary algorithm is designed and applied to group columns and find the optimum column family schema for the given user queries. The reading performance of the optimised column family schema is evaluated on a real dataset provided by ZANOX. It is shown that by using the found optimised column family schema, the reading performance while executing OLAP queries is improved with a statistical significance. Queries from a test set show that the average response time is reduced by up to 72% compared to reference column family schemas.
Keywords: HBase; NoSQL; column families; column layout; column grouping; evolutionary algorithms; MapReduce; multi-chromosome; schema optimisation; OLAP queries; Hadoop.
DOI: 10.1504/IJBDI.2017.081195
International Journal of Big Data Intelligence, 2017 Vol.4 No.1, pp.23 - 35
Received: 04 Jul 2015
Accepted: 26 Jan 2016
Published online: 26 Dec 2016 *