Title: Quasi-continuous maximum entropy distribution approximation with kernel density

Authors: Thomas Mazzoni, Elmar Reucher

Addresses: Department of Statistics, University of Hagen Profilstr. 8, Hagen, 58084, Germany. ' Department of Operations Research, University of Hagen, Profilstr. 8, Hagen, 58084, Germany

Abstract: This paper extends maximum entropy estimation of discrete probability distributions to the continuous case. This transition leads to a non-parametric estimation of a probability density function, preserving the maximum entropy principle. Furthermore, the derived density estimate provides a minimum mean integrated square error. In the second step, it is shown how boundary conditions can be included, resulting in a probability density function obeying maximum entropy. The criterion for deviation from a reference distribution is the Kullback-Leibler entropy. It is further shown, how the characteristics of a particular distribution can be preserved by using integration kernels with mimetic properties.

Keywords: maximum entropy; MaxEnt; Kullback-Leibler entropy; kernel density estimation; mean integrated spare error; mimetic properties.

DOI: 10.1504/IJIDS.2011.043026

International Journal of Information and Decision Sciences, 2011 Vol.3 No.4, pp.335 - 350

Published online: 30 Oct 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article