Significance of non-parametric statistical tests for comparison of classifiers over multiple datasets
by Pawan Kumar Singh; Ram Sarkar; Mita Nasipuri
International Journal of Computing Science and Mathematics (IJCSM), Vol. 7, No. 5, 2016

Abstract: In machine learning, generation of new algorithms or, in most cases, minor amendment of the existing ones is a common task. In such cases, a rigorous and correct statistical analysis of the results of different algorithms is necessary in order to select the exact technique(s) depending on the problem to be solved. The main inconvenience related to this necessity is the absence of proper compilation of statistical techniques. In this paper, we propose the use of two important non-parametric statistical tests, namely, Wilcoxon signed rank test for comparison of two classifiers and Friedman test with the corresponding post-hoc tests for comparison of multiple classifiers over multiple datasets. We also introduce a new variant of non-parametric test known as Scheffe's test for locating unequal pairs of means of performances of multiple classifiers when the given datasets are of unequal sizes. The parametric tests, which were previously being used for comparing multiple classifiers, have also been described in brief. The proposed non-parametric tests have also been applied on the classification results on ten real-problem datasets taken from the UCI Machine Learning Database Repository (http://www.ics.uci.edu/mlearn) (Valdovinos and Sanchez, 2009) as case studies.

Online publication date: Tue, 01-Nov-2016

The full text of this article is only available to individual subscribers or to users at subscribing institutions.

 
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.

Pay per view:
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.

Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Computing Science and Mathematics (IJCSM):
Login with your Inderscience username and password:

Username:        Password:          Forgotten your password?

Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.

If you still need assistance, please email subs@inderscience.com