Title: Comparison of statistical and machine learning methods in modelling of data with multicollinearity

Authors: Akhil Garg; Kang Tai

Addresses: School of Mechanical and Aerospace Engineering, Nanyang Technological University, 50 Nanyang Ave, Singapore 639798, Singapore ' School of Mechanical and Aerospace Engineering, Nanyang Technological University, 50 Nanyang Ave, Singapore 639798, Singapore

Abstract: Multicollinearity occurs in a dataset due to correlation between the predictors. Models derived from such data without a check on multicollinearity may lead to erroneous system analysis. This problem can be eliminated by the selection of appropriate predictors from the dataset. Variable reduction methods like B2, B4, VIF, KIF and factor analysis (FA) can be used to overcome this problem. Such methods are useful particularly when used in conjunction with modelling methods that do not automate variable selection, such as artificial neural network (ANN) and fuzzy logic. The literature reveals that the current problem is aptly described in the field of statistics but is paid little attention in the field of machine learning. In this paper, multicollinearity is presented involving the estimation of fat content inside the body. Commonly used statistical methods such as stepwise regression, radial basis function partial least squares, partial robust M-regression, ridge regression and principal component regression are applied to this problem. The machine learning methods FA-ANN and genetic programming are also applied. The results are discussed with the interpretation and comparison of the modelling methods summarised in order to guide users on the proper techniques for tackling the multicollinearity problem.

Keywords: multicollinearity; factor analysis; statistics; regression; genetic programming; artificial neural networks; ANNs; machine learning; principal component analysis; PCA; data modelling; fat content; body fat.

DOI: 10.1504/IJMIC.2013.053535

International Journal of Modelling, Identification and Control, 2013 Vol.18 No.4, pp.295 - 312

Published online: 16 Aug 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article