Finite Sample Based Mutual Information

dc.contributor.authorRajab, Khairan
dc.contributor.authorKamalov, Firuz© 2013
dc.descriptionThis article is not available at CUD collection. The version of scholarly record of this article is published in IEEE Access (2021), available online at:
dc.description.abstractMutual information is a popular metric in machine learning. In case of a discrete target variable and a continuous feature variable the mutual information can be calculated as a sum-integral of weighted log likelihood ratio of joint and marginal density distributions. However, in practice the true density distributions are unavailable and only a finite sample of the population is given. In this paper, we propose a novel method for calculating the mutual information for continuous variables using a finite sample of the population. The proposed method is based on approximating the underlying continuous density distribution using Kernel Density Estimation. Unlike previous kernel-based approaches for estimating mutual information, our method calculates directly the integral involved in the formula. Numerical experiments demonstrate that the proposed method produces more accurate results than the currently used feature selection approaches. In addition, our method demonstrates substantially faster computation times than the benchmark methods. © 2013 IEEE.en_US
dc.description.sponsorshipNajran University / NU/-/SERC/10/631en_US
dc.identifier.citationRajab, K., & Kamalov, F. (2021). Finite sample based mutual information. IEEE Access, 9, 118871-118879.
dc.publisherInstitute of Electrical and Electronics Engineers Inc.en_US
dc.relationAuthors Affiliations : Kamalov, F., Department of Electrical Engineering, Canadian University of Dubai, Dubai, UAE. (e-mail:; Rajab, K., College of Computer Science and Information System, Najran University, Najran, KSA.
dc.relation.ispartofseriesIEEE Access;Volume 9
dc.rightsCreative Commons Attribution 4.0 International (CC BY 4.0) License
dc.rights.holderCopyright : © 2013 IEEE.
dc.subjectContinuous variableen_US
dc.subjectfeature evaluationen_US
dc.subjectfeature selectionen_US
dc.subjectfinite sampleen_US
dc.subjectkernel density estimationen_US
dc.subjectmutual informationen_US
dc.titleFinite Sample Based Mutual Informationen_US
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
2.01 KB
Item-specific license agreed upon to submission