# Questions tagged [divergence]

a function that establishes the "distance" of one probability distribution to the other on a statistical manifold.

23 questions
Filter by
Sorted by
Tagged with
24 views

### What is the range of $a$ such that $\frac{logN}{N^{3a+1/2}}\rightarrow 0$ as $N\rightarrow \infty$? [migrated]

I want to find the range of $a$ such that $\frac{logN}{N^{3a+1/2}}\rightarrow 0$ as $N\rightarrow\infty$. The answer to this question hinges on conditions under which $logN$ diverges slower than the ...
41 views

### What are the advantages of Wasserstein distance compared to Jensen-Shannon divergence?

What is the practical difference between Wasserstein metric and Jensen-Shannon divergence? Wasserstein metric is also referred to as Earth mover's distance. From Wikipedia: Wasserstein metric is a ...
10 views

### Strong data processing inequality in multiplicative channels

We know that postprocessing will not increase the information. For two random variables $X$ and $Y$, $D(X||Y)>= D(f(X)||f(Y))$ for any operation $f()$ and divergence $D$. A strong data processing ...
55 views

41 views

### Sensitivity of KL Divergence

I am very new to the concept of KL divergence. Although I have grasped the fundamental formulations, I have a confusion comparing the KL divergence across the different distributions. Suppose I have 3 ...
25 views

### Does minimizing KL-divergence result in maximum entropy principle?

The Kullback-Leibler divergence (or relative entropy) is a measure of how a probability distribution differs from another reference probability distribution. I want to know what connection it has to ...
22 views

### Is limiting density of discrete points (LDDP) equivalent to negative KL-divergence?

Is limiting density of discrete points (LDDP), which is a corrected version of differential entropy, equivalent to the negative KL-divergence (or relative entropy) between a density function $m(x)$ ...
48 views

### Is relative entropy equal to cross-entropy during optimization?

I came across a saying that estimates of KL divergence, otherwise known as relative entropy, of the truth of a random variable and its prediction ($y$ and $\hat{y}$) is equal to their cross entropy ...
13 views

### Power-density divergence for logistic regression

How is power-density (or $\beta$-divergence) written for the binary logistic regression problem?
23 views

### Show that a sequence of random variables diverges to infinity in probability

I have sequences of real-valued random variables $\{X_T\}, \{Y_T\}$ and a sequence of real numbers $\{a_T\}$. As $T\rightarrow\infty$, I know that $$a_T \rightarrow \infty$$ and  X_T \overset{d}{\...
44 views

### Understanding concrete dropout

I am trying to understand a technique call Concrete Dropout for automatically tuning the dropout rate during the network training. However, I am unable to follow the work described in the paper ...
26 views

### What are the best known techniques to verify that a GAN samples correctly from a given distribution?

I would like to know what are the best known techniques to check that a generative adversarial network (GAN) samples from the correct distribution. Naively, I would say it all boils down to a ...
21 views

### Lemma KL-Divergence (Differential Privacy)

I am studying differential privacy and I got stuck again in proof of a lemma. Which is: "$D_{\infty}^\delta(Y||Z) \leq \epsilon$ if and only if there exists a random variable $Y'$ such that \$\...