I have a time series and wish to estimate its entropy, is it correct to use ksdensity function in Matlab to get estimates of the pdf at 100 points and then plug these into the standard Shannon entropy formula?

  • 2
    $\begingroup$ Since a density is not a probability, the answer surely is no. $\endgroup$
    – whuber
    Dec 9, 2016 at 17:12
  • $\begingroup$ Any advice on how to estimate entropy of a time series or where to find the right info? $\endgroup$
    – seanysull
    Dec 9, 2016 at 18:25

1 Answer 1


Depends on which Shannon entropy formula you use. Broadly, $H = \mathrm{E}[\mathrm{I}(X)]$. In the discrete case, this would be $H = \mathrm{E}[-\ln P(X)]$ or $$-\sum \limits_{i=1}^{n} P(x_i)\ln P(x_i)$$ This would make sense if you're using a histogram estimate of $P(X)$.

The kernel density estimate generates an estimate of the (continuous) density function $f(X)$. A such, you'd need a continuous estimate of $H = \mathrm{E}[\mathrm{I}(X)] = \mathrm{E}[-\ln f(X)]$ or $$-\int f(x)\ln f(x) dx$$ Since the kernel density estimate outputs discrete points $x_i, f(x_i)$, it would make sense to use something like trapezoid or Simpson's rule to get the integral.

I recommend consulting other sources (such as this one) to determine the best approximation method.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.