Timeline for What is the role of the logarithm in Shannon's entropy?
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| S Aug 19 at 18:34 | history | suggested | cottontail | CC BY-SA 4.0 |
replaced the dead link to the pdf version by its archival version
|
| Aug 19 at 17:28 | review | Suggested edits | |||
| S Aug 19 at 18:34 | |||||
| Apr 30, 2020 at 11:06 | comment | added | GENIVI-LEARNER | @NeilG I just saw your answer and commented there as it is more appropriate. | |
| Apr 29, 2020 at 23:27 | comment | added | Neil G | @GENIVI-LEARNER I explained why entropy is defined the way it is (instead of your definition, for example) in my answer. | |
| Apr 29, 2020 at 20:46 | comment | added | GENIVI-LEARNER | @NeilG [continued] distinguish the states for the coin toss apart. 1 bit has two states. So I can create arbitrary medium say Sixt = 6 states, then 1 Sixt of information is present in a dice just like 1 bit of information is in coin toss. It looks like we are taking long distance trip to just count the states by introducing the log. | |
| Apr 29, 2020 at 20:39 | comment | added | GENIVI-LEARNER | @NeilG I really like what Flounderer says that entropy is suppsed to measure different states of the system i.e $\Omega$, so why cant we simply define entropy by just no of states the system can take? i.e the entropy of dice shall be simply 6 and entropy of coin should be 2 etc. It makes intuitive sense. Why log transform? This transform makes sense if you use certain "medium" to tell the events apart say binary digit medium or bits but essentially why should be define entropy by creating some arbitrary medium and use that medium to tell the events apart.In binary medium 1 bit will [continued] | |
| S Apr 5, 2016 at 10:34 | history | edited | Richard Hardy | CC BY-SA 3.0 |
improved formatting, fixed typos
|
| S Apr 5, 2016 at 10:34 | history | suggested | Ooker | CC BY-SA 3.0 |
improve formatting
|
| Apr 5, 2016 at 10:20 | review | Suggested edits | |||
| S Apr 5, 2016 at 10:34 | |||||
| Feb 25, 2014 at 15:16 | comment | added | Neil G | This isn't why the log appears in the entropy calculation. This is why the information reported is reported as such. There is an alternative quantity: the "perplexity" that reports information without the log. In this part of his paper, Shannon is arguing in favor of bits/nats/hartleys, and against perplexity. | |
| Feb 20, 2014 at 1:41 | comment | added | bright-star | This answer seems to be the most focused yet informative. | |
| S Feb 19, 2014 at 22:38 | history | suggested | Piotr Migdal | CC BY-SA 3.0 |
wikipedia and pdf link for the cited paper
|
| Feb 19, 2014 at 22:35 | review | Suggested edits | |||
| S Feb 19, 2014 at 22:38 | |||||
| Feb 19, 2014 at 20:38 | history | answered | Flounderer | CC BY-SA 3.0 |