Title: Combining multiple classifiers for wrapper feature selection

Authors: Kyriacos Chrysostomou, Sherry Y. Chen, Xiaohui Liu

Addresses: School of Information Systems, Computing and Mathematics, Brunel University, Uxbridge, Middlesex, UB8 3PH, UK. ' School of Information Systems, Computing and Mathematics, Brunel University, Uxbridge, Middlesex, UB8 3PH, UK. ' School of Information Systems, Computing and Mathematics, Brunel University, Uxbridge, Middlesex, UB8 3PH, UK

Abstract: Wrapper feature selection methods are widely used to select relevant features. However, wrappers only use a single classifier. The downside to this approach is that each classifier will have its own biases and will therefore select very different features. In order to overcome the biases of individual classifiers, this study introduces a new data mining method called wrapper-based decision trees (WDT), which combines different classifiers and uses decision trees to classify selected features. The WDT method combines multiple classifiers so selecting classifiers for use in the combinations is an important issue. Thus, we investigate how the number and nature of classifiers influence the results of feature selection. Regarding the number of classifiers, results showed that few classifiers selected more relevant features whereas many selected few features. Regarding the nature of classifier, decision tree classifiers selected more features and the features that generated accuracies much higher than other classifiers.

Keywords: feature selection; wrappers; decision trees; support vector machine; SVM; Bayesian networks; multiple classifiers; data mining.

DOI: 10.1504/IJDMMM.2008.022539

International Journal of Data Mining, Modelling and Management, 2008 Vol.1 No.1, pp.91 - 102

Published online: 14 Jan 2009 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article