Title: Measuring harmfulness of class imbalance by data complexity measures in oversampling methods

Authors: Anjana Gosain; Anju Saha; Deepika Singh

Addresses: University School of Information, Communication and Technology, Guru Gobind Singh Indraprastha University, New Delhi, India ' University School of Information, Communication and Technology, Guru Gobind Singh Indraprastha University, New Delhi, India ' University School of Information, Communication and Technology, Guru Gobind Singh Indraprastha University, New Delhi, India

Abstract: Many real world applications consist of skewed datasets which result in class imbalance problem. During classification, class imbalance cause underestimation of minority classes. Researchers have proposed a number of algorithms to deal with this problem. But recent research studies have shown that some skewed datasets are unharmful and applying class imbalance algorithms on these datasets lead to degenerated performance and increased execution time. In this research paper, we have pre-estimated the degree of harmfulness of class imbalance for skewed classification problems, using two of the data complexity measures: scatter matrix based class separability measure and ratio of intra-class versus inter-class nearest neighbours. Also the performance of oversampling-based class imbalance classification algorithms have been analysed with respect to these data complexity measures. The experiments are conducted using k-nearest neighbour (k-nn) and naive Bayes as the base classifiers for this study. The obtained results illustrate the usefulness of these measures by providing the prior information about the nature of the imbalance datasets that help us to select the more efficient classification algorithm.

Keywords: class imbalance; data complexity measure; class separability measure; class overlapping; inter-class nearest neighbour; intra-class nearest neighbour; imbalance ratio; oversampling method.

DOI: 10.1504/IJIEI.2019.099089

International Journal of Intelligent Engineering Informatics, 2019 Vol.7 No.2/3, pp.203 - 230

Received: 03 Mar 2017
Accepted: 04 Oct 2017
Published online: 15 Apr 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article