17
$\begingroup$

I'm trying to make a video about loaded dice, and at one point in the video we roll about 200 dice, take all the sixes, roll those again, and take all the sixes and roll those a third time. We had one die that came up 6 three times in a row, which is obviously not unusual because there should be a 1/216 chance of that happening and we had about 200 dice. So how do I explain that it's not unusual? It doesn't quite seem like the Law of Large Numbers. I want to say something like "If you do enough tests, even unlikely things are bound to happen" but my partner said people might take issue with the "bound to" terminology.

Is there a standard way to state this concept?

Improve this question
$\endgroup$
9
  • 9
    $\begingroup$ en.wikipedia.org/wiki/Infinite_monkey_theorem $\endgroup$
    – James
    Aug 19, 2013 at 12:20
  • $\begingroup$ The probability p=1/n basically means that you have 1 success per n tirals. This is what it means and this is how it is checked. If you do not see 1 success per n experiments, you report us a wrong probability. Now, you say that n is large. But what is the difference when you also say that you can do much more experiments that n? I mean that you do not need any law besides the definition of probability. I am more interested to know why is the probability of having success in n trials is not 1? $\endgroup$
    – Val
    Aug 19, 2013 at 14:38
  • 3
    $\begingroup$ @Val Your comments have to be read in a peculiar way in order not to be misunderstood! When the probability of an event is $1/n$, it actually is likely that event will not be observed in $n$ independent trials. (The probability of not observing it is close to $1/e\approx 0.37$ for large $n$). So you seem to be wrong about your assertion concerning checking rare probabilities. I think you go wrong by conflating probabilities with frequencies: they definitely differ, both conceptually and in practice. $\endgroup$
    – whuber ♦
    Aug 19, 2013 at 14:47
  • $\begingroup$ My success = your observation. I do not understand why you started to reinterpret this precisely clear statement and redefine everything. Secondly, though I always believed that probability is something theoretial (computed combinatorically in probability theory) whereas frequency is its statistical (i.e. experimental) confirmation, the law of big numbers say that frequency converges to probability probability at large number of experiments and I see no reason to highlight the difference, at least in this case. $\endgroup$
    – Val
    Aug 19, 2013 at 15:08
  • 1
    $\begingroup$ I do not understand your last two comments. I am interpreting the words you use in what I believe are standard ways. In particular I am highlighting the fact that probability is not the same as an observed frequency, which is what your first sentence appears to say. When a probability is $1/n$, by the way, then $n$ is not a "large number of experiments" by any means: there will be large deviations between observed frequencies and underlying probabilities. This is not related to any consideration of duplicate values. $\endgroup$
    – whuber ♦
    Aug 19, 2013 at 17:05

5 Answers 5

Reset to default
19
$\begingroup$

Law of truly large numbers:

... with a sample size large enough, any outrageous thing is likely to happen.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I think it's pretty obvious this is the best answer here, lol. $\endgroup$
    – Phillip Schmidt
    Aug 19, 2013 at 13:03
  • 3
    $\begingroup$ This is the temporal version of the Totalitarian principle. $\endgroup$
    – Ray Koopman
    Aug 20, 2013 at 2:12
13
$\begingroup$

You could explain that even as an event specified a priori, the probability that it occurs is not low. Indeed, it's not so hard to calculate the probability of 3 or more rolls of sixes in a row for at least one die out of 200.

[Incidentally, there's a nice approximate calculation you can use - if you have $n$ trials there there's a probability of $1/n$ of a 'success' (for $n$ not too small), the chance of at least one 'success' is about $1-1/e$. More generally, for $kn$ trials, the probability is about $1-e^{-k}$. In your case you're looking at $m = kn$ trials for a probability of $1/n$ where $n=216$ and $m=200$, so $k = 200/216$, giving a probability of around 60% that you'll see 3 sixes in a row at least once out of the 200 sets of 3 rolls.

I don't know that this specific calculation has a particular name, but the general area of rare events with many trials is related to the Poisson distribution. Indeed the Poisson distribution itself is sometimes called 'the law of rare events', and even occasionally 'the law of small numbers' (with 'law' in these cases meaning 'probability distribution').]

--

However, if you didn't specify that particular event before the rolling and only say afterward 'Hey, wow, what are the chances of that?', then your probability calculation is wrong, because it ignores all the other events about which you'd say 'Hey, wow, what are the chances of that?'.

You've only specified the event after you observe it, for which 1/216 doesn't apply, even with only one die.

Imagine I have a wheelbarrow full of small, but distinguishable dice (maybe they have little serial numbers) - say I have ten thousand of them. I tip the wheelbarrow full of dice out:

die #    result
00001      4
00002      1
00003      5
 .         .
 .         .
 .         .
09999      6
10000      6

... and I go "Hey! Wow, what are the chances I'd get '4' on die #1 and '1' on die #2 and ... and '6' on die #999 and '6' on die #10000?"

That probability is $\frac{1}{6}^{10000}$ or about $3.07\times 10^{-7782}$. That's an astonishingly rare event! Something amazing must be going on. Let me try again. I shovel them all back in, and tip the wheelbarrow out again. Again I say "hey, wow, what are the chances??" and again it turns out I have an event of such astonishing rarity it should only happen once in the lifetime of a universe or something. What's up?

Simply, I am doing nothing but trying to calculate the probability of an event specified after the fact as if it had been specified a priori. If you do that, you get crazy answers.

Improve this answer
$\endgroup$
4
  • 17
    $\begingroup$ You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won't believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing! -- Richard Feynman. $\endgroup$
    – gerrit
    Aug 19, 2013 at 9:25
  • $\begingroup$ This is not what the OP is asking. This is more like the "Antrophic principle" (is there a more generic term for that?) while the term the OP is asking is more like the "law of truly large numbers"? $\endgroup$
    – Lie Ryan
    Aug 19, 2013 at 14:24
  • 3
    $\begingroup$ @LieRyan If the OP's question contains an implied reasoning error, to which an ordinary probability calculation should not be applied, it would be wrong not to point that out clearly. Indeed, even if there's just a good possibility that issue exists, it should be clearly pointed out. Since there was no hint that the event was in fact specified before the observation, it needs to be pointed out. The required detail to convey exactly why it's a problem takes more than a couple of sentences. I do speak to the direct question in my first paragraph, but then explain why there's a problem. $\endgroup$
    – Glen_b
    Aug 19, 2013 at 14:31
  • 2
    $\begingroup$ Just for clarification, it was a priori. $\endgroup$
    – Cassandra Gelvin
    Aug 19, 2013 at 19:31
3
$\begingroup$

I think that your statement "If you do enough tests, even unlikely things are bound to happen", would be better expressed as "If you do enough tests, even unlikely things are likely to happen". "bound to happen" is a bit too definite for a probability issue and I think the association of unlikely with likely in this context makes the point you are trying to put over.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I disagree, "bound to happen" is correct. Unless the dice is rigged to avoid the unlikely event, then it will happen. If it doesn't happen, then you just haven't done enough trials, either thator that it is not "unlikely things" but "impossible things". $\endgroup$
    – Lie Ryan
    Aug 19, 2013 at 14:11
  • 1
    $\begingroup$ Technically speaking, an event is only "bound to happen" if you try an infinite number of times; it's an asymptote. Probability has no memory; in theory I could flip a fair coin every second from now until the heat-death of the universe and only get heads. Taken as a whole, that's a very unlikely event, but each flip is still a 50/50 chance, so at no point does it become certain that I'll get tails. Likewise, even with a huge number of trials, that unlikely event is still just as unlikely for any given single trial - it might never happen. $\endgroup$
    – anaximander
    Aug 19, 2013 at 14:39
  • 1
    $\begingroup$ Of course, that assumes that you know the probabilities of your events. In the real world, after a certain number of trials you have to point out that your calculations give you a 99.999% chance of seeing the unlikely event at least once by now, and you still haven't seen it, so perhaps it's less likely than you thought (or maybe even impossible). $\endgroup$
    – anaximander
    Aug 19, 2013 at 14:41
  • 1
    $\begingroup$ @Anaximander A subtler interpretation of "bound to happen" that makes it a correct assertion about unlikely events is this: for all $0\le q\lt 1$ there exists an $n$ for which the probability of the event occurring in $n$ or more independent observations is at least $q$. This definition does not need to drag in some undefined or vague sense of "infinite number." In this sense any event of strictly positive probability $\varepsilon$ is bound to happen eventually: for the proof, just take $n \gt \log(1-q)/\log(1-\varepsilon)$ and do the (elementary) calculation. $\endgroup$
    – whuber ♦
    Aug 20, 2013 at 19:55
2
$\begingroup$

I think what you need is a zero-one law. The most famous of these is the Kolmogorov Zero-One Law, which states that any event in the event space we're interested in will either eventually occur with probability 1 or never occur with probability 1. That is to say, there is no grey area of events that may happen.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ I believe Kolmogorov's law applies only to tail events, not to "any event ... we're interested in." You might be able to apply this law to general events to shed light on the question, but some explanation of how to do that would be helpful here. $\endgroup$
    – whuber ♦
    Aug 20, 2013 at 19:59
  • $\begingroup$ This is a good comment: I think the precise definition of tail event is exactly what we're looking for to solve this. I'll do some research on it. $\endgroup$
    – owensmartin
    Aug 22, 2013 at 16:58
1
$\begingroup$

Glivenko–Cantelli theorem (Wikipedia Link)

This theorem says, loosely speaking, that as the number of samples grows, the empirical distribution tends toward ("converges") the true distribution.

In that sense, if there truly is a nonzero probability of an event happening, enough observations should lead to you seeing it happen, since your empirical CDF has to tend toward the true CDF that gives such an event positive, even if small, probability.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.