Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

I am studying a dynamical system that takes as an initial condition a list. I want to analyze the evolution of Shannon's entropy in this system. I know the maximum entropy (50) and the minimum (0). Pure random conditions have almost maximum entropy, and so it is hard to analyze changes in it unless it decreases. I set up the list to have an initial value of 25 (average between maximum and minimum), so there is an equal amount to expand in either direction. Is this statistically sound?

Thanks in advance.

share|improve this question
But what is the question precisely? Can you give more details? – mbq♦ Jan 28 '11 at 8:02
What equation do you use to calculate the entropy? Like a "frequency" entropy? What constraints are the maximised entropy based on? agree with @mbq, this question needs more details. – probabilityislogic Jan 28 '11 at 16:13
I use this: upload.wikimedia.org/math/a/2/f/…. It is probability-based, as it compares the frequency of a state to the probability of a state. The maximum entropy is simply the highest possible entropy for the list, and occurs when the frequency for the two states is equal. The minimum occurs when it is all one state. The specific question is whether it would be "correct" to set the frequencies in the initial condition to produce an entropy that is an average between the minimum and maximum, to allow for equal change in entropy in either direction. – user2976 Jan 28 '11 at 21:10
Hang on, to have maximum entropy of $50$, this means you have $n=e^{50}=gozillian$ categories? is that right? and how many times will you observe the system? – probabilityislogic Jan 29 '11 at 17:44
No, a list of length 100 with 2 states is enough. $50=-\frac{50*0.5*\log{0.5}}{\log{2}}-\frac{50*0.5*\log{0.5}}{\log{2}}$ – user2976 Jan 29 '11 at 18:49
show 4 more comments

1 Answer

up vote 1 down vote

The "expected entropy" at a particular time step given a particular starting point is a well-defined quantity and you could certainly study it. There is no particular reason why you should favor the "median" entropy without knowing anything else about the system. You should do experiments with as diverse of a set of starting configurations as possible to get a better understanding of your system.

share|improve this answer
Also recall that any measurement of entropy are estimates. Therefore, they have an underlying distribution if the processes that give rise to it are random! – Néstor Jul 24 '12 at 0:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.