Timeline for Is it true that entropy estimation is meaningless if samples are not i.i.d.?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 6, 2021 at 20:41 | vote | accept | Mark | ||
| May 6, 2021 at 16:14 | answer | added | Sweetm | timeline score: 1 | |
| May 6, 2021 at 15:56 | answer | added | mhdadk | timeline score: 1 | |
| May 6, 2021 at 12:56 | comment | added | Mark | Actually, I have a time series I would like to estimate Shannon's entropy about. I wonder if this series should be considered a realization of an i.i.d. process to obtain a proper entropy's estimation | |
| May 6, 2021 at 12:24 | comment | added | mhdadk | Could you be more specific in your question about what $\hat{H}$ is? It could be $$\hat{H} = -\frac{1}{N} \sum_{i=1}^N \log{p(x_i)},$$ but there is no way to tell. | |
| May 6, 2021 at 12:22 | comment | added | Mark | @mhdadk: I mean that, if $\hat{H}$ can be defined in terms of frequencies (i.e., counts) $f\left(x_{n}\right)$ which go to replace $p\left(x_{n}\right)$ in $H(X)$, then $\hat{H}$ is meaningless if $x_{n}$ are not i.i.d.. | |
| May 6, 2021 at 12:18 | comment | added | mhdadk | What do you mean by "$x_n$ are not i.i.d"? In this context, $x_n$ are not samples from a distribution, they are the range of $X$. | |
| May 6, 2021 at 10:55 | history | asked | Mark | CC BY-SA 4.0 |