I want to calculate the Kullback Leibler divergence between a multivariate t distribution and a multivariate normal distribution, for different values of the degrees of freedom $\nu$.

However, this requires a multiple integration that seems to be difficult to calculate numerically for dimensions larger than 2. Is there a known result to calculate this integral or a numerical trick?

I understand there are general multivariate numerical integration methods. I was just wondering if there is a simpler ad hoc tool I could use as these are popular distributions, so I guess there may be some simpler tools.

  • $\begingroup$ Monte Carlo integration? $\endgroup$ – seanv507 Feb 7 at 12:17
  • $\begingroup$ @seanv507 Thanks, that is too general to be of any use, but I appreciate the pointer. $\endgroup$ – Pullback Feb 7 at 13:20
  • $\begingroup$ t-distribution is a scale mixture of Gaussian distributions and fining KL divergence to mixtures is notoriously difficult. You can lower bound on it if you are interested. $\endgroup$ – passerby51 Feb 7 at 15:13

There is a numerical solution based on one-dimensional numerical integrals here:

Kullback Leibler divergence between a multivariate t and a multivariate normal distributions

enter image description here

I doubt there is a closed form solution, but the 1D numerical integral seems simple.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.