3
$\begingroup$

I want to calculate the Kullback–Leibler divergence between a multivariate $t$ distribution and a multivariate normal distribution, for different values of the degrees of freedom $\nu$.

However, this requires a multiple integration that seems to be difficult to calculate numerically for dimensions larger than 2. Is there a known result to calculate this integral or a numerical trick?

I understand there are general multivariate numerical integration methods. I was just wondering if there is a simpler ad hoc tool I could use as these are popular distributions, so I guess there may be some simpler tools.

Improve this question
$\endgroup$
4
  • $\begingroup$ Monte Carlo integration? $\endgroup$
    – seanv507
    Commented Feb 7, 2021 at 12:17
  • $\begingroup$ @seanv507 Thanks, that is too general to be of any use, but I appreciate the pointer. $\endgroup$
    – Pullback
    Commented Feb 7, 2021 at 13:20
  • $\begingroup$ t-distribution is a scale mixture of Gaussian distributions and fining KL divergence to mixtures is notoriously difficult. You can lower bound on it if you are interested. $\endgroup$
    – passerby51
    Commented Feb 7, 2021 at 15:13
  • $\begingroup$ This paper has what you are looking for. The final solution is on page 8 and is quite cumbersome. link.springer.com/content/pdf/10.1007/s40304-019-00200-8.pdf $\endgroup$
    – Snowy Baboon
    Commented Oct 31, 2023 at 1:08

1 Answer 1

Reset to default
2
$\begingroup$

There is a numerical solution based on one-dimensional numerical integrals here:

Kullback Leibler divergence between a multivariate t and a multivariate normal distributions

enter image description here

I doubt there is a closed form solution, but the 1D numerical integral seems simple.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.