I'm trying to calculate the Jensen-Shannon divergence between two multivariate Gaussians.

I found a closed-form expression both for the KL divergence and JS divergence between two Gaussians in this paper: Here

(Sorry I'm new to stackexchange so I don't know how to type in math equations)

  • For KL divergence between two Gaussians: See equation (76) on page 12

  • For JS divergence between two Gaussians: See equation (83) on page 13

I implemented these two equations in matlab as follows:

function gJS = gaussian_gJS(d, mu1, sigma1, mu2, sigma2)

alpha = 0.5;

sigma1_inv = inv(sigma1);
sigma2_inv = inv(sigma2);

M1_inv         = inv(-0.5*sigma1_inv);
M2_inv         = inv(-0.5*sigma2_inv);

Sigma_a = inv( (1 - alpha)*sigma1_inv + alpha*sigma2_inv );
Mu_a    = Sigma_a * ( (1 - alpha)*M1_inv*mu1 + alpha*M2_inv*mu2 );

gJS = (1 - alpha)*gaussian_KL(d, mu1, sigma1, Mu_a, Sigma_a) + ...
            alpha*gaussian_KL(d, mu2, sigma2, Mu_a, Sigma_a);

function KL_Divergence = gaussian_KL(d, mu1, sigma1, mu2, sigma2)

inv_sigma2 = inv(sigma2);

KL_Divergence = 0.5*trace(inv_sigma2*sigma1) + (mu2 - mu1)'*inv_sigma2 + log(det(sigma2)/det(sigma1)) - d;


In the paper, the author gives a numerical example (on page 13, below equation 84) where he calculates the gJS for some example:

 gJS( mu_1 = -1, sigma_1 = 4, mu_2 = -4, sigma_2 = 16) = 0.4087

However when I calculate it using my implementation, I end up with a value of around 67.94.

One good thing is that my function is symmetric around inputs so at least that part is working. I've double checked my implementation, and I don't see any differences between my code and his equations. Further, if I calculate the KL-divergence using my function, for a Gaussian against itself, I don't get zero, instead I get -0.5. I've seen another thread on stackexchange where they derived the KL divergence between two multivariate Gaussians and their expression was slightly different than Eqn. (76) in the paper I linked.

Anyone have any thoughts?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.