Linked Questions
17 questions linked to/from KL divergence between two multivariate Gaussians
1
vote
0
answers
265
views
KL divergence between two multivariate gaussians where $p$ is $N(\mu, I)$ [duplicate]
We know if we try to get $D_{KL}(q||p)$, where $p$ is a standard normal distribution, so mean is 0, variance is the identity matrix, and $q$ is a multivariate normal distribution, it can be calculated ...
- 285
152
votes
2
answers
191k
views
KL divergence between two univariate Gaussians
I need to determine the KL-divergence between two Gaussians. I am comparing my results to these, but I can't reproduce their result. My result is obviously wrong, because the KL is not 0 for KL(p, p).
...
- 13.7k
29
votes
1
answer
19k
views
Deriving the KL divergence loss for VAEs
In a VAE, the encoder learns to output two vectors:
$$\mathbf{\mu} \in\ \mathbb{R}^{z}$$
$$\mathbf{\sigma} \in\ \mathbb{R}^{z}$$
which are the mean and variances for the latent vector $\mathbf{z}$, ...
- 1,221
5
votes
2
answers
9k
views
How to use Kullback-leibler divergence if mean and standard deviation of of two Gaussian Distribution is provided?
With Apache Spark MLLib library I am trying to find Clusters within a dataset by using Gaussian Mixture Model (number cluster =3) . Now it returns 3 different values of mean and standard deviation. I ...
- 51
7
votes
1
answer
2k
views
KL divergence for joint probability distributions?
I have a pair of joint probability distributions. I want to measure their similarity/dissimilarity. If they were single-dimensional probability distributions, then I could measure the Kullback–Leibler ...
- 2,932
5
votes
3
answers
2k
views
Combine overlapping Gaussian distributions
Given the mean and covariance matrix of the pair-wise (multivariate normal) distributions
\begin{align}
P(X_1,X_2) &\sim \mathcal{N}(\mu_{12},\Sigma_{12}),\\
P(X_2,X_3) &\sim \mathcal{N}(\mu_{...
- 213
6
votes
2
answers
2k
views
textbook example of KL Divergence [duplicate]
I have read what KL Divergence is about: assess differences in probability distributions between two sets.
I have also read, and digested, that it is emphatically not a true metric because of ...
- 8,747
3
votes
1
answer
3k
views
KL divergence between two bivariate Gaussian distribution
KL divergence between two multivariate Gaussians and univariate Gaussians have been discussed. I was wondering if there exists a simpler computation for the KL divergence between two bivariate ...
- 33
4
votes
0
answers
3k
views
KL divergence between two multivariate Gaussians with close means and variances
KL divergence between two Gaussian distributions denoted by $\mathcal{N}(\mathbf \mu_1, \mathbf \Sigma_1)$ and $\mathcal{N}(\mathbf \mu_2, \mathbf \Sigma_2)$ is available in a closed form as:
$$\...
- 41
0
votes
0
answers
2k
views
Kernel Density Estimation - Comparison Between different sets of samples
Is there a way for compare the distribution of different set of samples?
For example, I have three sets, for example:
X1 = N(0, 1);
X2 = N(0.5, 1);
X3 = N(1, 1).
Each set is drown with a specific (...
- 101
1
vote
1
answer
802
views
Can multivariate Gaussians KL divergence be a negative value?
I'm trying to find if two hidden neurons in RBF Network overlap with each other or not? It's an online classification problem, it means data come to our network one-by-one and then discard completely. ...
- 267
2
votes
1
answer
438
views
Expectation of multivariate gaussian w.r.t. other multivariate gaussian
I want to calculate the Kullbach Leibler Divergence of two multivariate Gaussians as in
KL divergence between two multivariate Gaussians.
At one point one has to solve the following expression (...
- 767
2
votes
1
answer
302
views
Computing KL divergence between uniform and multivariate Gaussian
Another post has addressed the fact that KL divergence is defined between a uniform distribution and a Gaussian distribution $$D_{\text{KL}}(\mathcal{U}(x) \parallel \mathcal{N}(x \mid \mu, \Sigma)) = ...
- 611
1
vote
0
answers
308
views
skew G-Jensen-Shannon divergence between multivariate gaussian calculation discrepancy
I'm trying to calculate the Jensen-Shannon divergence between two multivariate Gaussians.
I found a closed-form expression both for the KL divergence and JS divergence between two Gaussians in this ...
1
vote
1
answer
110
views
How to soften or mitigate vector similarity measure?
I would like to evaluate a similarity between two objects X and Y by comparing a neighbourhood in which they're located.
I construct two sets of nine concentric and equidistant circles with centers ...
- 1,114