7
$\begingroup$

I would like to quantify the amount of uncertainty in a given message, but the signal I work with is non-stationary and non-linear.

Is it possible to apply Shannon entropy for such signal?

$\endgroup$
  • 1
    $\begingroup$ "Possible" in what sense? $\endgroup$ – JohnRos Oct 29 '11 at 23:04
  • 2
    $\begingroup$ Do you know the exact stochastic process, or do you only have observations? Can you describe your signal in details? Do you want the entropy rate as a function of time? Do you have independent trials? $\endgroup$ – Memming Dec 16 '13 at 15:50
7
$\begingroup$

Shannon Entropy is a concept related to the distribution of a random variable, not to any particular realization of the r.v. The OP talks about a "non-stationary" signal. This implies that the OP has available a sequence of signals, which can be viewed as a realized sequence of a stochastic process, which is a sequence of random variables.

If the process was (strictly) stationary then each r.v. would have the same distribution, hence the same entropy, and the specific realization of the process (the data) could be used to form some estimate of this common entropy.

If the stochastic process is not strictly stationary, then each element-random variable of the process may have a different entropy. In that case the theoretical validity of the Entropy concept remains -but if non-stationarity is left totally unrestricted, then we do not have a sufficient amount of data to estimate these different entropies.

This is a general issue with non-stationary stochastic process, it affects estimation attempts of all measures, characteristics, moments, statistics etc related to such a process. If we do not somehow restrict the memory and the time-heterogeneity of the process, we won't have enough data to say anything about it.

So any question about Shannon Entropy and non-stationary data should include the assumed restrictions on non-stationarity (assumed based on theory and/or on data assessment), in order to be actually answerable.

$\endgroup$
0
$\begingroup$

I would examine the windowed entropy and empirical PDF/CDF to see how rapidly the signal is changing, and whether it is an issue.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.