# Questions tagged [kullback-leibler]

An asymmetric measure of distance (or dissimilarity) between probability distributions. It might be interpreted as the expected value of the log likelihood ratio under the alternative hypothesis.

434 questions
Filter by
Sorted by
Tagged with
0answers
19 views

0answers
28 views

### Binary cross entropy and relationship with cross entropy function or KL divergence

I am learning GAN right now and I don't understand why the cost of the original GAN function will cause mode clapse and many problems. I did some research and notice it has something to do with KL ...
1answer
63 views

### Kullback-Leibler divergence between multivariate t and the multivariate normal?

I want to calculate the Kullback Leibler divergence between a multivariate t distribution and a multivariate normal distribution, for different values of the degrees of freedom $\nu$. However, this ...
0answers
34 views

### KL divergence and Wasserstein distance

While reading the paper https://arxiv.org/pdf/1903.11780.pdf, I have some confusion parts as below: The KL divergence is not only problematic for representation learning due to the statistical ...
1answer
33 views

### JS-divergence vs KL-divergence as objective for GAN?

I would love to get some help with that interview question I failed to answer correctly: In what sense is the JS-divergence prefferable over KL-divergence $D_{KL}(p_{data}||p_{model})$ as an objective ...
1answer
20 views

### Proving that JSD is symmetric?

How can I prove that the Jensen–Shannon divergence (https://en.wikipedia.org/wiki/Jensen%E2%80%93Shannon_divergence) is symmetric? Thanks!
1answer
55 views

### How can I prove that KL-divergence is not symmetric?

How can I prove that KL-divergence is not symmetric? https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence Thanks a lot!
1answer
31 views

### How do measure how different two policies are?

I have two agents that both follow a baseline behavioral policy pi(a|s). If I then modify the state-action distribution for the two agents (resulting in two new policies), is there a standard measure ...
0answers
37 views