Title: Rule extraction from support vector machines: a hybrid approach for solving classification and regression problems
Author: Mohammad Abdul Haque Farquad, Vadlamani Ravi, S. Bapi Raju
Institute for Development and Research in Banking Technology, Castle Hills Road #1, Masab Tank, Hyderabad – 500 057 (AP), India; Department of Computer and Information Sciences, University of Hyderabad, Hyderabad – 500 046 (AP), India.
Institute for Development and Research in Banking Technology, Castle Hills Road #1, Masab Tank, Hyderabad – 500 057 (AP), India.
Department of Computer and Information Sciences, University of Hyderabad, Hyderabad – 500 046 (AP), India
Journal: Int. J. of Information and Decision Sciences, 2011 Vol.3, No.3, pp.265 - 294
Abstract: In this paper, a novel hybrid approach to extract rules from support vector machine and support vector regression (SVM/SVR) is presented. The hybrid has three phases: 1) SVM-recursive feature elimination (RFE) algorithm is employed for feature selection; 2) using the selected features, SVM/SVR models are built and the actual target values of the training instances are replaced by the predictions obtained from these models resulting in a modified training set; 3) the modified training set is used for rule generation using decision tree (DT), classification and regression tree (CART), adaptive network based fuzzy inference system (ANFIS) and dynamic evolving fuzzy inference system (DENFIS). Extensive experiments are conducted on three benchmark classification problems, four bank bankruptcy prediction problems and five benchmark regression problems. We conclude that the rules obtained after feature selection perform comparably to those extracted from all features. Further, comprehensibility is also improved after feature selection.
Keywords: rule extraction; support vector machines; SVM; support vector regression; SVR; decision trees; classification; regression trees; CART; adaptive neuro-fuzzy inference system; ANFIS; dynamic evolving fuzzy inference system; DENFIS; root mean squared error; RMSE; feature selection; modelling; bank bankruptcy prediction; neural networks; fuzzy logic.