2
$\begingroup$

Given the KL divergence value between 2 distributions, how should someone use this to determine whether the value is significant for the distributions $P$ and $Q$ to be different? One method I can speculate is that a Monte Carlo sample of the CDF of the $P$ distribution can produce a range of KL distances in which there is a pvalue set at the border for the 5% largest values.

From the motivation and background for the G-test, wiki page, it shows the connection between KL divergence and the Chisquared test and the G-test. There is a clear pvalue for the Chi squared test, in that it can be applied for hypothesis testing. Is there an equivalent for KL divergence (or even the G-test)?

EDIT: the situation is that $P$ and $Q$ are sampled distributions and that they both contain long consistent tails.

$\endgroup$
2
  • $\begingroup$ If you have distributions $p$ and $q$ and the KL divergence is nonzero, then $p$ and $q$ are different. There's no need to think about significance, hypothesis testing, etc. unless you're estimating the distributions from sampled values. If that's the case, perhaps you could clarify the situation you have in mind. $\endgroup$
    – user20160
    Commented Sep 14, 2018 at 22:57
  • 1
    $\begingroup$ Why do you want to funnel your statistical comparison of 2 samples, possibly drawn from the same distribution, through KLD instead of using a direct statistical test? What problem are you trying to solve, and how does KLD help you solve it? stats.stackexchange.com/questions/83163/… $\endgroup$
    – Sycorax
    Commented Feb 17, 2021 at 8:04

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.