Title: Bias reduction via linear combination of nearest neighbour entropy estimators

Authors: Alexei Kaltchenko, Nina Timofeeva

Addresses: Department of Physics and Computer Science, Wilfrid Laurier University, Waterloo, Ontario N2L 3C5, Canada. ' Department of Computer Science, Yaroslavl State University, Yaroslavl, 150000, Russia

Abstract: The problem of entropy estimation of stationary ergodic processes is considered. A new family of entropy estimators is constructed as a linear combination of the nearest neighbour estimators with a new metric. The consistency of the new estimators is established for the broad class of measures. The O (n−b)-efficiency of these estimators is established for symmetric probability measures, where b > 0 is a constant and n is the number of observations.

Keywords: entropy rate; information source; stationary ergodic stochastic processes; entropy estimation; bias reduction; nearest neighbour estimators.

DOI: 10.1504/IJICOT.2009.024046

International Journal of Information and Coding Theory, 2009 Vol.1 No.1, pp.39 - 56

Published online: 24 Mar 2009 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article