Title: The extended Kullback-Leibler divergence measure in the unknown probability density function cases and applications

Authors: Hoa Le; Hoang Van Truong; Pham The Bao

Addresses: University of Economics and Law, Vietnam National University Ho Chi Minh City (VNU-HCM), Ho Chi Minh City, Vietnam ' University of Science, Vietnam National University Ho Chi Minh City (VNU-HCM), Ho Chi Minh City, Vietnam ' Sai Gon University, Ho Chi Minh City, 700000, Vietnam

Abstract: The Kullback-Leibler divergence measure is used to evaluate the similarity between two probability distributions. In theory, the probability density functions are known before applying the formula. However, estimating this information of real data is challenging. For that reason, the Kullback-Leibler divergence needs to be modified for similarity measures in these cases. In this paper, we proposed and evaluated an extended Kullback-Leibler divergence similarity measure by two experiments. The first experiment is based on two datasets that have unknown probability density functions, while the second one is conducted on one dataset with an unknown probability density function and the other with a known probability density function. Besides, the proposed method is applied to the simulated data and the plagiarism detection cases.

Keywords: Kullback-Leibler divergence; relative entropy; mixture models; similarity measure; extended Kullback-Leibler divergence.

DOI: 10.1504/IJIIDS.2021.118555

International Journal of Intelligent Information and Database Systems, 2021 Vol.14 No.4, pp.403 - 420

Received: 26 Aug 2020
Accepted: 04 Feb 2021
Published online: 28 Oct 2021 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article