All Questions

6 questions
Filter by
Sorted by
Tagged with
498 views

Upper bound on KL divergence

Is there a maximum (unique?) to the KL divergence between discrete distributions p & q, with the restriction that q is a proper probability distribution? I know KL is unbounded from above when q ...
In Cover & Thomas, Elements of Information Theory, at the section on Conditional Limit Theorem (11.6), it is proved that the KL divergence bounds the $\cal{L}_1$-norm from above, $\frac{1}{2\ln2}\... 0answers 203 views Asymmetry of the Kullback-Leibler distance in hypothesis testing My question is related to the asymmetry of the Kullback-Leibler distance. I'm using the discrete definition of the Kullback-Leibler distance, so we have:$$KL(p,q) = \sum_{s \in S} p(s) \log\left( \... 0answers 265 views Kullback Leibler divergence “efficient” upper bound For a distribution of N values, how can I efficiently upper-bound the largest divergence between all non-negative distributions over the same random field? For example, for all distributions of a ... 1answer 171 views How does one express the decrease in minimal type II error bound for each observation added? Problem: I have a "classifier" that uses some arbitrary hypothesis test on observations from one of two known probability distributions:$P_0$(null hypothesis$H_0$) is a zero-mean Gaussian$\...
In my research I have run into the following general problem: I have two distributions $P$ and $Q$ over the same domain, and a large (but finite) number of samples from those distributions. Samples ...