Linked Questions

1 vote
0 answers
286 views

KL divergence between two multivariate gaussians where $p$ is $N(\mu, I)$ [duplicate]

We know if we try to get $D_{KL}(q||p)$, where $p$ is a standard normal distribution, so mean is 0, variance is the identity matrix, and $q$ is a multivariate normal distribution, it can be calculated ...
Gergő Horváth's user avatar
161 votes
2 answers
208k views

KL divergence between two univariate Gaussians

I need to determine the KL-divergence between two Gaussians. I am comparing my results to these, but I can't reproduce their result. My result is obviously wrong, because the KL is not 0 for KL(p, p). ...
bayerj's user avatar
  • 13.8k
30 votes
1 answer
22k views

Deriving the KL divergence loss for VAEs

In a VAE, the encoder learns to output two vectors: $$\mathbf{\mu} \in\ \mathbb{R}^{z}$$ $$\mathbf{\sigma} \in\ \mathbb{R}^{z}$$ which are the mean and variances for the latent vector $\mathbf{z}$, ...
YellowPillow's user avatar
  • 1,261
6 votes
2 answers
10k views

How to use Kullback-leibler divergence if mean and standard deviation of of two Gaussian Distribution is provided?

With Apache Spark MLLib library I am trying to find Clusters within a dataset by using Gaussian Mixture Model (number cluster =3) . Now it returns 3 different values of mean and standard deviation. I ...
Avik Dutta's user avatar
7 votes
1 answer
2k views

KL divergence for joint probability distributions?

I have a pair of joint probability distributions. I want to measure their similarity/dissimilarity. If they were single-dimensional probability distributions, then I could measure the Kullback–Leibler ...
rhombidodecahedron's user avatar
7 votes
2 answers
2k views

textbook example of KL Divergence [duplicate]

I have read what KL Divergence is about: assess differences in probability distributions between two sets. I have also read, and digested, that it is emphatically not a true metric because of ...
cgo's user avatar
  • 9,217
5 votes
3 answers
2k views

Combine overlapping Gaussian distributions

Given the mean and covariance matrix of the pair-wise (multivariate normal) distributions \begin{align} P(X_1,X_2) &\sim \mathcal{N}(\mu_{12},\Sigma_{12}),\\ P(X_2,X_3) &\sim \mathcal{N}(\mu_{...
Manuel Schmidt's user avatar
3 votes
1 answer
3k views

KL divergence between two bivariate Gaussian distribution

KL divergence between two multivariate Gaussians and univariate Gaussians have been discussed. I was wondering if there exists a simpler computation for the KL divergence between two bivariate ...
Phoenix666's user avatar
4 votes
0 answers
3k views

KL divergence between two multivariate Gaussians with close means and variances

KL divergence between two Gaussian distributions denoted by $\mathcal{N}(\mathbf \mu_1, \mathbf \Sigma_1)$ and $\mathcal{N}(\mathbf \mu_2, \mathbf \Sigma_2)$ is available in a closed form as: $$\...
Dave's user avatar
  • 41
0 votes
0 answers
2k views

Kernel Density Estimation - Comparison Between different sets of samples

Is there a way for compare the distribution of different set of samples? For example, I have three sets, for example: X1 = N(0, 1); X2 = N(0.5, 1); X3 = N(1, 1). Each set is drown with a specific (...
Luca's user avatar
  • 101
1 vote
1 answer
904 views

Can multivariate Gaussians KL divergence be a negative value?

I'm trying to find if two hidden neurons in RBF Network overlap with each other or not? It's an online classification problem, it means data come to our network one-by-one and then discard completely. ...
mkafiyan's user avatar
  • 267
2 votes
1 answer
418 views

Computing KL divergence between uniform and multivariate Gaussian

Another post has addressed the fact that KL divergence is defined between a uniform distribution and a Gaussian distribution $$D_{\text{KL}}(\mathcal{U}(x) \parallel \mathcal{N}(x \mid \mu, \Sigma)) = ...
adamconkey's user avatar
2 votes
1 answer
456 views

Expectation of multivariate gaussian w.r.t. other multivariate gaussian

I want to calculate the Kullbach Leibler Divergence of two multivariate Gaussians as in KL divergence between two multivariate Gaussians. At one point one has to solve the following expression (...
guest1's user avatar
  • 991
1 vote
0 answers
325 views

skew G-Jensen-Shannon divergence between multivariate gaussian calculation discrepancy

I'm trying to calculate the Jensen-Shannon divergence between two multivariate Gaussians. I found a closed-form expression both for the KL divergence and JS divergence between two Gaussians in this ...
Little Finger's user avatar
1 vote
1 answer
110 views

How to soften or mitigate vector similarity measure?

I would like to evaluate a similarity between two objects X and Y by comparing a neighbourhood in which they're located. I construct two sets of nine concentric and equidistant circles with centers ...
Adam Przedniczek's user avatar

15 30 50 per page