Timeline for Differential Entropy drops when any random variable is normalized to unit variance
Current License: CC BY-SA 3.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 23, 2017 at 14:01 | comment | added | James Bowery | I thought that entropy is obviously different from Kolmogorov complexity when one considers pseudo-random number generators. | |
| Aug 26, 2013 at 18:22 | history | edited | Scortchi♦ | CC BY-SA 3.0 |
fixed typos
|
| Feb 19, 2013 at 9:16 | comment | added | Pat | @Cagdas - I dunno if I'd call it a gimmick. It's just measuring a different thing. And as cardinal points out, it has some uses. As for whether it'll break when applied to the binominal distribution, well, depends how you're going to apply it :). Probably worth starting a new topic if you're not sure. | |
| Feb 19, 2013 at 9:14 | comment | added | Pat | @Cardinal. Yeah, I knew that $\log(d x)$ was a horribly odd thing to talk about when I was writing it. However I think going about it in this manner helps really drive home why differential entropy really really isn't entropy. | |
| Feb 18, 2013 at 18:23 | comment | added | cardinal | The notation $\log(\mathrm d x)$ is not really very meaningful, but we can turn some of your exposition into something a little more precise. Indeed, if the density $p(x)$ is Riemann integrable, then $-\sum_{i} p(x_i) \delta x \log p(x_i) \to h(X)$ as $\delta x \to 0$. An interpretation of this that you will often see is that an $n$-bit quantization of a continuous random variable has entropy of about $h(X) + n$. | |
| Feb 18, 2013 at 18:10 | comment | added | Cagdas Ozgenc | Thanks. That's very interesting. I didn't know there was such a gimmick in the theory. | |
| Feb 18, 2013 at 18:08 | vote | accept | Cagdas Ozgenc | ||
| Feb 18, 2013 at 17:23 | history | answered | Pat | CC BY-SA 3.0 |