Timeline for What is the role of the logarithm in Shannon's entropy?
Current License: CC BY-SA 4.0
19 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| S May 4, 2020 at 10:42 | history | suggested | tinlyx | CC BY-SA 4.0 |
improved language and fixed typo
|
| May 4, 2020 at 3:25 | review | Suggested edits | |||
| S May 4, 2020 at 10:42 | |||||
| Apr 29, 2020 at 19:50 | comment | added | GENIVI-LEARNER | @whuber I have pretty much same concern on entropy. Why take log transformation. How does logarithm quantify diversity and why variance cant be used in place of entropy? large variance means large disorder and variance uses square transformation and not log transform. I am quite intrigued by having to define entropy by the number of bits we need to tell the events apart. Why not simply count the total number of events or take average "amount" of events in a system to quantify diversity? So by that definition a coin has "2 diversity" and dice has "6 diversity" | |
| Oct 20, 2019 at 7:20 | history | edited | Nick Cox | CC BY-SA 4.0 |
deleted 1 character in body
|
| Sep 28, 2018 at 3:37 | review | Suggested edits | |||
| Sep 28, 2018 at 7:10 | |||||
| Apr 13, 2017 at 12:58 | history | edited | CommunityBot |
replaced http://mathoverflow.net/ with https://mathoverflow.net/
|
|
| Mar 25, 2015 at 13:57 | history | edited | Piotr Migdal | CC BY-SA 3.0 |
link updated
|
| Dec 17, 2014 at 23:03 | history | edited | Piotr Migdal | CC BY-SA 3.0 |
formula simpler and more general at the same time
|
| Feb 21, 2014 at 3:01 | comment | added | histelheim | whuber - I think @PiotrMigdal's first comment tells what "entropy without the logarithm is" - diversity. Maybe this doesn't make mathematical sense, but it makes intuitive sense - the measures are similar, but not the same. | |
| Feb 21, 2014 at 2:57 | comment | added | histelheim | @ Piotr Migdal - further, your explanation following "We can call log(1/p) information. Why?" seems to make sense to me. Is it that the logarithm essentially moves us from a diversity index to an information index - measuring the number of bits we need to tell the events apart. | |
| Feb 21, 2014 at 2:51 | comment | added | histelheim | @ Piotr Migdal - you write "logarithm is to make it growing linearly with system size and "behaving like information"." - this seems crucial for me to understand the role of the logarithm, however I'm not quite clear as to what it means. | |
| Feb 20, 2014 at 12:24 | history | edited | Piotr Migdal | CC BY-SA 3.0 |
misc
|
| Feb 20, 2014 at 12:13 | history | edited | Piotr Migdal | CC BY-SA 3.0 |
expanded with two coin example
|
| Feb 19, 2014 at 23:09 | history | edited | Piotr Migdal | CC BY-SA 3.0 |
gram misc
|
| Feb 19, 2014 at 21:23 | comment | added | whuber♦ | I am baffled by your last comment, Histelheim: what could "entropy without the logarithm" possibly refer to? That suggests you haven't yet clearly articulated your question, because it sounds like you have some unstated concept of "entropy" in mind. Please don't keep us guessing--edit your question so that your readers can provide the kinds of answers you are looking for. | |
| Feb 19, 2014 at 19:51 | comment | added | Piotr Migdal | @histelheim What you mean by "without the logarithm"? $\sum_i p_i$ is just one. If you want another measure of diversity without $\log$, look at diversity indices - e.g. so-called Inverse Simpson index $1/\sum_i p_i^2$ which tells effective number of choices (one over average probability), there is Gini–Simpson index $1-\sum_i p_i^2$ which is always between 0 and one. And if you don't care for subtle information-related properties of Shannon entropy, you can use any of them (though, they weight low and high probabilities differently). | |
| Feb 19, 2014 at 19:40 | comment | added | histelheim | This answer has a lot of nice details - but from a layman's perspective it still skirts the issue - what is the role of the logarithm? Why can't we calculate entropy without the logarithm? | |
| Feb 19, 2014 at 18:39 | history | edited | Piotr Migdal | CC BY-SA 3.0 |
coding theorem misc
|
| Feb 19, 2014 at 18:33 | history | answered | Piotr Migdal | CC BY-SA 3.0 |