Title: Query optimisation in real-time data warehouses

Authors: Issam Hamdi; Emna Bouazizi; Jamel Feki

Addresses: MIRACL Laboratory, University of Sfax, Sfax 3018, Tunisia ' MIRACL Laboratory, University of Sfax, Sfax 3018, Tunisia ' Faculty of Computing and IT, University of Jeddah, Jeddah, Saudi Arabia; MIRACL Laboratory, University of Sfax, Sfax 3018, Tunisia

Abstract: A real-time data warehouse (RTDW) allows decision makers to analyse fresh data as fast as possible in order to support real-time decision processes. In this paper, we focus on optimisation techniques to speed up query processing; in particular, we propose a dynamic selection of materialised views algorithm (DynaSeV) which selects views from results of incoming queries. Secondly, we suggest a new update policy to dynamically maintain materialised views. In addition, we propose a novel data partitioning approach for RTDW, called 2LPA-RTDW (Two-Level data Partitioning Approach for RTDW) by allowing unbalance of data amount in each partition. Then, we present our architecture called DETL-(m, k)-firm-RTDW architecture (decentralised extract-transform-load approach based on (m, k)-Firm constraints for RTDW) which deals with diversity and disparities in data source systems to reduce the time for ETL. Finally, we evaluate our contributions using the TPC-DS (TPC, 2014) benchmark; the preliminary results are quite promising.

Keywords: real-time data warehouse; RTDW; real-time transactions; materialised views; data partitioning; extract-transform-load; ETL.

DOI: 10.1504/IJIIDS.2019.104534

International Journal of Intelligent Information and Database Systems, 2019 Vol.12 No.4, pp.245 - 278

Accepted: 21 Apr 2019
Published online: 17 Jan 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article