Title: Confidence intervals for the mutual information

Authors: Arno G. Stefani; Johannes B. Huber; Christophe Jardin; Heinrich Sticht

Addresses: Institute for Information Transmission (LIT), FAU Erlangen-Nuremberg, Cauerstr. 7, 91058 Erlangen, Germany ' Institute for Information Transmission (LIT), FAU Erlangen-Nuremberg, Cauerstr. 7, 91058 Erlangen, Germany ' Bioinformatics, Institute for Biochemistry, FAU Erlangen-Nuremberg, Fahrstr. 17, 91054 Erlangen, Germany ' Bioinformatics, Institute for Biochemistry, FAU Erlangen-Nuremberg, Fahrstr. 17, 91054 Erlangen, Germany

Abstract: By combining a bound on the absolute value of the difference of mutual information between two joint probability distributions with a fixed variational distance, and a bound on the probability of a maximal deviation in variational distance between a true joint probability distribution and an empirical joint probability distribution, confidence intervals for the mutual information of two random variables with finite alphabets are established. Different from previous results, these confidence intervals do not need any assumptions on the distribution or the sample size.

Keywords: mutual information; nonparametric estimation; variational distance; classification; Bayes error estimation; sample size; k-means clustering; confidence intervals; joint probability distributions.

DOI: 10.1504/IJMISSP.2014.066430

International Journal of Machine Intelligence and Sensory Signal Processing, 2014 Vol.1 No.3, pp.201 - 214

Received: 12 Oct 2013
Accepted: 07 Feb 2014
Published online: 31 Dec 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article