Timeline for Why am I getting information entropy greater than 1?
Current License: CC BY-SA 3.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 20, 2019 at 18:47 | comment | added | Nick Cox | Categories that don't occur in practice have observed probability zero and don't affect the entropy result. It's a strong convention, which can be justified more rigorously, that $-0 \log 0$ is to be taken as zero (the base of logarithms being immaterial here). | |
| Aug 20, 2019 at 18:41 | comment | added | eljusticiero67 | just to clarify, is it k the number of possible categories, or number of categories youre calculating entropy for? eg. i have 10 possible categories, but there are 3 samples representing 2 categories in the system i am calculating entropy for. is k in this case 2? | |
| Jul 20, 2018 at 11:49 | comment | added | Nick Cox | The length of the sample is immaterial too. It's how many categories you have. | |
| Jul 20, 2018 at 11:44 | comment | added | Fierce82 | didn't know about that, thanks. So basically the base of logarithm thas to be the same as the length of the sample, and not more? | |
| Apr 13, 2016 at 12:07 | history | answered | Nick Cox | CC BY-SA 3.0 |