Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

For two $n$ dimensional multivariate normal distributions $X_{1}\sim N\left(\mu_{1},\Sigma_{1}\right)$ and $X_{2}\sim N\left(\mu_{2},\Sigma_{2}\right)$, the Kullback-Leibler distance is given by $$KL=\frac{1}{2}\left(tr\left(\Sigma_{2}^{-1}\Sigma_{1}\right)+\left(\mu_{2}-\mu_{1}\right)'\Sigma_{1}^{-1}\left(\mu_{2}-\mu_{1}\right)-ln\left(\frac{det\left(\Sigma_{1}\right)}{det\left(\Sigma_{2}\right)}\right)-n\right) $$

When $\Sigma_{1}=\Sigma_{2}$, the above will be similar to half the Mahalanobis distance squared. The Mahalanobis distance squared is chi-square distributed with $n-1$ degrees of freedom, so I imagine that is part of the answer.

When the parameters of $KL$ are replaced with estimates from on a random sample, is it possible to derive the distribution in the more general case when $\Sigma_{1}$ is not necessarily equal to $\Sigma_{2}$?

share|improve this question
KL is not a random variable. Are you interested in the distribution of KL when the parameters are replaced with their estimates based on a random sample ? Your question should be precised. – Stéphane Laurent Sep 28 '12 at 20:31
Yes, that is what I am interested in. Made some changes to the question, hope it makes more sense. – John Sep 28 '12 at 20:39

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.