Title: Optimisations of four imputation frameworks for performance exploring based on decision tree algorithms in big data analysis problems

Authors: Jale Bektaş; Turgay Ibrikci

Addresses: Department of Computer Technology and Information Systems, School of Applied Technology and Management, Mersin University, Erdemli, Mersin, 33730, Turkey ' Software Engineering Department, Adana Alparslan Turkes Science and Technology University, Adana, Turkey

Abstract: The phenomenon of how to treat missing values is a problem confronted in big data analysis. Therefore, various applications have been developed on imputation strategies. This study focused on four imputation frameworks proposing novel perspectives based on expectation-maximisation (EM), self-organising map (SOM), K-means and multilayer perceptron (MLP). Initially, several transformation steps such as normalised, standardised, interquartile range and wavelet were applied. Then, imputed datasets were analysed using decision tree algorithms (DTAs) by optimising their parameters. These analyses showed that DTAs had not been strikingly affected by any data transformation techniques except interquartile range. Even though the dataset contains a missing value ratio of 33.73%, the EM imputation framework provided a performance increase of 0.42% to 3.14%. DTAs based on C4.5 and NBTree algorithms have been more stable for all big imputed datasets. Furthermore, realistic performance measurement of any preprocessing experiment based on C4.5 can be proposed to avoid time complexity.

Keywords: preprocessing; data mining; multiple imputation; decision tree classifier; machine learning; big data analytics.

DOI: 10.1504/IJCSE.2022.126256

International Journal of Computational Science and Engineering, 2022 Vol.25 No.5, pp.523 - 531

Received: 24 May 2021
Accepted: 18 Sep 2021
Published online: 18 Oct 2022 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article