# The operation of chance in a deterministic world

In Steven Pinker's book Better Angels of Our Nature, he notes that

Probability is a matter of perspective. Viewed at sufficiently close range, individual events have determinate causes. Even a coin flip can be predicted from the starting conditions and the laws of physics, and a skilled magician can exploit those laws to throw heads every time. Yet when we zoom out to take a wide-angle view of a large number of these events, we are seeing the sum of a vast number of causes that sometimes cancel each other out and sometimes align in the same direction. The physicist and philosopher Henri Poincare explained that we see the operation of chance in a deterministic world either when a large number of puny causes add up to a formidable effect, or when a small cause that escapes our notice determines a large effect that we cannot miss. In the case of organized violence, someone may want to start a war; he waits for the opportune moment, which may or may not come; his enemy decides to engage or retreat; bullets fly; bombs burst; people die. Every event may be determined by the laws of neuroscience and physics and physiology. But in the aggregate, the many causes that go into this matrix can sometimes be shuffled into extreme combinations. (p. 209)

I am particularly interested in the bolded sentence, but I give the rest for context. My question: are there statistical ways of describing the two processes that Poincare described? Here are my guesses:

1) "A large number of puny causes add up to a formidable effect." The "large number of causes" and "add up" sound to me like the central limit theorem. But in (the classical definition of) the CLT, the causes need to be random variables, not deterministic effects. Is the standard method here to approximate these deterministic effects as some sort of random variable?

2) "A small cause that escapes our notice determines a large effect that we cannot miss." It seems to me like you could think of this as some sort of hidden Markov model. But the (unobservable) state transition probabilities in an HMM are just that, probabilities, which is by definition once again not deterministic.

-

Interesting thought (+1).

In cases 1) and 2), the problem is the same: we do not have complete information. And probability is a measure of the lack of information.

1) The puny causes may be purely deterministic, but which particular causes operate is impossible to know by a deterministic process. Think of molecules in a gaz. The laws of mechanics apply, so what is random here? The information hidden to us: where is which molecule with what speed. So the CLT applies, not because there is randomness in the system, but because there is randomness in our representation of the system.

2) There is a time component in HMM that is not necessarily present in this case. My interpretation is the same as before, the system may be non random, but our access to its state has some randomness.

EDIT: I don't know if Poincare was thinking of a different statistical approach for these two cases. In case 1) we know the varialbes, but we cannot measure them because there are too many and they are too small. In case 2) we don't know the variables. Both ways, you end up making assumptions and modeling the observable the best we can, and quite often we assume Normality in case 2).

But still, if there was one difference, I think it would be emergence. If all systems were determined by sums of puny causes then all random variables of the physical world would be Gaussian. Clearly, this is not the case. Why? Because scale matters. Why? Because new properties emerge from interactions at smaller scale, and these new properties need not be Gaussian. Actually, we have no statistical theory for emergence (as far as I know) but maybe one day we will. Then it will be justified to have different statistical approaches for cases 1) and 2)

-
Thanks for the answer. Agree both come down to the fact that we do not have complete information--that is a good way of framing it. However, I'd like to see an answer that distinguishes between the two cases more. What was Poincare thinking? –  Andy McKenzie Jun 2 '12 at 5:27
I see you concern. I edited my answer to try and answer the best I can. –  gui11aume Jun 2 '12 at 9:32

I think you are reading too much into the statement. It all seems to lie under the premise that the world is deterministic and that humans model it probabilistically because it is easier to approximate what is going on that way than to go through all the details of the physics and any other mathematical equations that describe it. I think that there has been a long standing debate about determininism versus random effects particularly between physicist and statisticians. I was particularly struck by the following preceding sentences to what you bolded. "Even a coin flip can be predicted from the starting conditions and the laws of physics, and a skilled magician can exploit those laws to throw heads every time." When I was a graduate student at Stanford in the late 1970s Persi Diaconis a statistician and a magician and Joe Keller a physicist actually tried to apply the laws of physics to a coin flip to determine what the otucome would be based on the intial conditions regarding whether or not heads is face up and exact;y how the force of the finger flip strikes the coin. i think they may have worked it out. But to think a magician even with the magical training and statistical knowledge of a persi diaconis could flip the coin and have it come up heads every time is preposterous. I believe they found that it is impossible to replicate the initial conditions and I think chaos theory applies. Small perturbations in the initial condition have large effects on the flight of the coin and make the outcome unpredictable. As a statistician I would say even if the world is deterministic stochastic models do a better job of predicting outcomes than complex deterministic laws. When the physics is simple deterministic laws can and should be used. For example Newton's gravitational law works well at determining the speed that an object has when it hits the ground being dropped from 10 feet above the ground and using the equation d=gt$^2$/2 you can solve for the time it takes to complete the fall very accurately as well since the gravitational constant g has been determined to a high level of accuracy and the equation applies almost exactly.

-
Michael Chernick, you might be interested in this article about Diaconis. –  Cyan Jun 1 '12 at 23:32
I would replace the sentence "...humans model it probabilistically because it is easier to approximate what is going on that way..." with "...humans model it probabilistically because it is too difficult to incorporate the tiny details, which most of the time don't matter,...". Additionally, you are taking a "practical" approach to a more philosophical/conceptual question. Chaos theory is only a problem "in practice" because we do not have arbitrarily accurate representations of numbers. Another problem with deterministic laws is they often depend on things we can't measure. –  probabilityislogic Jun 1 '12 at 23:48
Thanks Cyan. I haven't seen that particular article but I have seen several others about Persi and i knwo him pretty well as a former assitant professor who taught me probability theory and time series when we were both in our late twenties and earlier thirties from 1974-1978. Also Persi had me and Michael Cohen (when Michael Cohen and I were both graduate students) role shaved dice on a cloth hundreds or thousands of times to confirm his theory as to what the bias would be for that type of shaving. –  Michael Chernick Jun 1 '12 at 23:50
Like any good experimenter he did not tell us that they were shaved and it was not so big of a difference in area to make it noticeable to the eye. of course if you wanted to cheat a gambling establishment with shaved dice you could not shave so much to make it noticeable and yet not so little that it would take you forever to gain some good winnings and avoid the gamblers ruin. Of ocurse we had some suspicion about the experiment because it didn't make much sense to try to confirm that each side came up very close to 1/6 th of the time. –  Michael Chernick Jun 1 '12 at 23:55
Also doing an experient to show that you can bias a fair coin in favor of heads is far from being able to get a head every time. Statisticians are used by lottery commissions to test their machines to make sure that they are fair. –  Michael Chernick Jun 1 '12 at 23:57

earlier version had an incorrect $2^{-N}$ term in the first equation, and was missing a $2^N$ in the third equation. Thanks to Cardianl for noting this

This is not a full answer, but too long for a comment. Just to give some mathematical intuition about point 1) we have the following limit in large $N$ (using stirling's approximation, so $a_n \sim b_n$ means $\lim_{n\to\infty}\frac{a_n}{b_n}=1$)

$${N\choose Nf}\sim\sqrt{\frac{1}{2\pi Nf(1-f)}}\exp\left(-NH(f)\right)$$

Where $H(f)=f\log\left(f\right)+(1-f)\log\left(1-f\right)$ is the entropy function. We also have second order Taylor series for $H(f)$ (about the mode of $\frac{1}{2}$) of:

$$H(f)\approx-\log\left(2\right)+2(f-\frac{1}{2})^2$$

So we also have:

$${N\choose Nf}\sim 2^N\sqrt{\frac{1}{2\pi Nf(1-f)}}\exp\left(-\frac{2}{N}(Nf-\frac{N}{2})^2\right)$$

The meaning of these limits is that any procedure which consists of counting the possibile ways in which something could happen (such as causal-effect analysis) is lead to the normal distribution. This does not depend on $f$ being random or deterministic. What the central limit theorem says is that the majority of ways in which a given set of events could happen is well approximated by a normal distribution.

-
Thank you. I guess the OP was not reading too much into connecting the bolded sentence with the CLT. But can I make sure I understand this correctly? Are you saying that for large N the number of combinations of N things taken Nf at a time is approximately equal to the normal density with variance parameter Nf(1-F) and mean parameter N/2? Also this is just an asymptotic mathematical property with no connection to probability? That is as amazing as seeing the De Moivre - LaPlace version of the central limit theorem in action using the quincunx devise! –  Michael Chernick Jun 2 '12 at 2:34
Thanks, it is very helpful to think about the normal distribution non-probabilistically. However, I don't understand 1) how that first limit arises and 2) what point you are doing the Taylor series expansion around. –  Andy McKenzie Jun 2 '12 at 6:03
Can you clarify what your $a_n \sim b_n$ notation means? It cannot be the standard asymptotic notation indicating $a_n / b_n \to 1$. The left hand side is huge and the right hand side is tiny! Maybe once this is sorted out, it will not surprise @Michael so much since convergence in distribution (and even a local limit law like this) is simply an analytical statement about certain sequences of monotonically nondecreasing functions and so, at its heart, is not tied to any "probabilistic notions". –  cardinal Jun 2 '12 at 14:44
The edits are looking better. There must still be a missing term in the first display equation, though. :) –  cardinal Jun 3 '12 at 3:04
@cardinal - the first equation is correct, by substituting stirlings approximation for each factorial. Terms involving $\log(N)$ cancel inside the exponential. –  probabilityislogic Jun 3 '12 at 21:44
show 1 more comment

The quote from Pinker's book and the idea of a deterministic world completely ignore Quantum Mechanics and the Heisenberg Uncertaintly Principle. Imagine putting a small amount of something radioactive near a detector and arranging the amounts and distances so that there will be a 50% chance of detecting a decay during a pre-determined time interval. Now connect the detector to a relay that will do something highly significant if a decay is detected and operate the device once and only once.

You have now created a situation where the future is inherently unpredictable. (This example is drawn from one described by whomever taught sophomore or junior year physics at MIT back in the middle 1960's.)

-