# Tagged Questions

The tag has no wiki summary.

4answers
2k views

### An adaptation of the Kullback-Leibler distance?

Look at this picture: If we draw a sample from the red density then some values are expected to be less than 0.25 whereas it is impossible to generate such a sample from the blue distribution. As a ...
2answers
1k views

### Hypothesis testing and total variation distance vs. Kullback-Leibler divergence

In my research I have run into the following general problem: I have two distributions $P$ and $Q$ over the same domain, and a large (but finite) number of samples from those distributions. Samples ...
2answers
2k views

### Kullback-Leibler divergence - interpretation

I have a question about the Kullback-Leibler divergence. Can someone explain why the "distance" between the blue density and the "red" density is smaller than the distance between the "green" curve ...
2answers
5k views

### KL divergence between two univariate Gaussians

I need to determine the KL-divergence between two Gaussians. I am comparing my results to these, but I can't reproduce their result. My result is obviously wrong, because the KL is not 0 for KL(p, p). ...
3answers
3k views

### Questions about KL divergence?

I am comparing two distributions with KL divergence which returns me a non-standardized number that, according to what I read about this measure, is the amount of information that is required to ...
3answers
3k views

### Measure of similarity or distance between two symmetric covariance matrices

are there any measures of similarity or distance between two symmetric covariance matrices (both having the same dimension)? I am thinking here of analogues to KL divergence of two probability ...
1answer
1k views

### Kullback–Leibler vs Kolmogorov-Smirnov distance

I can see that there are a lot of formal differences between Kullback–Leibler vs Kolmogorov-Smirnov distance measures. However, both are used to measure the distance between distributions. Is there ...
2answers
156 views

1answer
98 views

### How does one express the decrease in minimal type II error bound for each observation added?

Problem: I have a "classifier" that uses some arbitrary hypothesis test on observations from one of two known probability distributions: $P_0$ (null hypothesis $H_0$) is a zero-mean Gaussian ...