I would like to quantify the amount of uncertainty in a given message, but the signal I work with is non-stationary and non-linear.
Is it possible to apply Shannon entropy for such signal?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
Shannon Entropy is a concept related to the distribution of a random variable, not to any particular realization of the r.v. The OP talks about a "non-stationary" signal. This implies that the OP has available a sequence of signals, which can be viewed as a realized sequence of a stochastic process, which is a sequence of random variables.
If the process was (strictly) stationary then each r.v. would have the same distribution, hence the same entropy, and the specific realization of the process (the data) could be used to form some estimate of this common entropy.
If the stochastic process is not strictly stationary, then each element-random variable of the process may have a different entropy. In that case the theoretical validity of the Entropy concept remains -but if non-stationarity is left totally unrestricted, then we do not have a sufficient amount of data to estimate these different entropies.
This is a general issue with non-stationary stochastic process, it affects estimation attempts of all measures, characteristics, moments, statistics etc related to such a process. If we do not somehow restrict the memory and the time-heterogeneity of the process, we won't have enough data to say anything about it.
So any question about Shannon Entropy and non-stationary data should include the assumed restrictions on non-stationarity (assumed based on theory and/or on data assessment), in order to be actually answerable.