Timeline for How to interpret KL divergence quantitatively?
Current License: CC BY-SA 2.5
13 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 3, 2011 at 3:22 | history | edited | Yaroslav Bulatov | CC BY-SA 2.5 |
added 20 characters in body
|
| Aug 13, 2010 at 21:25 | comment | added | Yaroslav Bulatov | Actually a much easier to understand reference for this is Cover's "Elements of Information Theory", page 309, 12.8 "Stein's Lemma" | |
| Aug 13, 2010 at 1:59 | history | edited | Yaroslav Bulatov | CC BY-SA 2.5 |
added 209 characters in body; added 20 characters in body
|
| Aug 13, 2010 at 1:42 | history | edited | Yaroslav Bulatov | CC BY-SA 2.5 |
added 9 characters in body
|
| Aug 12, 2010 at 22:59 | history | edited | Yaroslav Bulatov | CC BY-SA 2.5 |
added 4 characters in body
|
| Aug 12, 2010 at 22:54 | comment | added | Yaroslav Bulatov | ok, we don't need small e (now called b, Type II error) to be small for bound to hold, but b=0 is the value for which the simplified (exp(-n KL(p,q)) bound matches the more complicated bound above. Curiously enough, lower bound for Type I error given 0 Type II error is <1, I wonder if <1 Type II error rate is actually achievable | |
| Aug 12, 2010 at 22:08 | history | edited | Yaroslav Bulatov | CC BY-SA 2.5 |
added 14 characters in body
|
| Aug 12, 2010 at 21:23 | history | edited | Yaroslav Bulatov | CC BY-SA 2.5 |
added 631 characters in body; deleted 17 characters in body
|
| Aug 12, 2010 at 20:07 | comment | added | Yaroslav Bulatov | Why do I take small e...hmm...that's what Balasubramanian's paper did, but now, going back to Kullback, it seems his bound holds for any e, and he also gives bound for finite n, let me update the answer | |
| Aug 12, 2010 at 11:08 | comment | added | robin girard | +1 I like this interpretation! could you clarify " p below e "? why do you take small e ? you say "the probability of making the opposite mistake is" it is an upper bound or exact probability? If I remember, this type of approach is due to Chernoff, do you have the references (I find your first reference is not clarifying the point :) ) ? | |
| Aug 11, 2010 at 23:53 | history | edited | Yaroslav Bulatov | CC BY-SA 2.5 |
added 78 characters in body; added 6 characters in body
|
| Aug 11, 2010 at 23:17 | history | edited | Yaroslav Bulatov | CC BY-SA 2.5 |
added 1 characters in body
|
| Aug 11, 2010 at 23:09 | history | answered | Yaroslav Bulatov | CC BY-SA 2.5 |