Timeline for Why do we weight the surprisals by the probabilities when computing the entropy?
Current License: CC BY-SA 4.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 1, 2023 at 18:16 | comment | added | JoeTheShmoe | @tim this is a pretty circular argument in your comment, I'd say a better question op could have asked is "why do we need those assumptions Claude mentioned, and what would average information achieve conceptually" | |
| Jun 26, 2023 at 15:39 | comment | added | Tim | @MehdiCharife Yes, I understand, but it is inconsistent with Shannon's requirements. If this metric is useful for you for some reason, feel free to use it, but it is not what we mean by entropy. | |
| Jun 26, 2023 at 15:35 | comment | added | Mehdi Charife | In the formula I mentioned the sum is divided by $n$ (the number of unique values taken by $X$ i.e. $Card(X(\Omega))$. Different $x_i$ could have the same probability $p$, thus if we wanted to re-write the sum as a sum of unique probability values, each surprisal $-log(p)$ would be weighted by the number of $x_i$ having the same probability $p$ divided by $n$. | |
| Jun 26, 2023 at 14:55 | history | answered | Tim | CC BY-SA 4.0 |