Title: A guide to statistical analysis for single criteria heuristics evaluation

Authors: Belarmino Adenso-Diaz, Norberto Corral

Addresses: Engineering School, Universidad de Oviedo, Spain. ' Department of Statistics, Universidad de Oviedo, Spain

Abstract: The success obtained by metaheuristic techniques in resolving combinatory problems of a real character has led to a veritable explosion in the number of studies in this field. From a scientific perspective, these new approaches must demonstrate improved performance, normally with respect to prior proposals. This involves the carrying out of a series of computational experiments and statistical tests demonstrating their validity. However, in a large number of cases and for diverse causes (a lack of data relative to the benchmark heuristic or due to not knowing the most appropriate statistical tests), some authors fail to present conclusive statistical tests that may be qualified as possessing a minimum level of rigour according to the scientific method. This paper analyses the different tests that may be considered to verify significant differences between the performance of different single criteria heuristics, as well as including a number of application examples, and a discussion about the problem of robustness.

Keywords: heuristics analysis; statistical tests; algorithmics; single criteria heuristics.

DOI: 10.1504/IJICA.2007.016791

International Journal of Innovative Computing and Applications, 2007 Vol.1 No.2, pp.94 - 107

Published online: 22 Jan 2008 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article