If I flip a fair coin 10 times, and all of them land on heads, according to the Law of Large Numbers I am more likely to get tails on the next flip. However, this is clearly the Gambler's Fallacy.
How am I misinterpreting the Law of Large Numbers?
$\begingroup$
$\endgroup$
4
-
1$\begingroup$ Because it does not say what you think it says. The Law of Large Numbers is a statement about limits, and as such it is unaffected by any initial finite pattern. $\endgroup$– HenryCommented Jul 26, 2016 at 19:06
-
$\begingroup$ Are you saying that the law of large numbers says nothing about how fast the probabilities should approach the expected? $\endgroup$– Fine-TuningCommented Jul 26, 2016 at 19:09
-
1$\begingroup$ See Does 10 heads in a row increase the chance of the next toss being a tail? and this one adds some further intuition to that: Statistics concept to explain why you're less likely to flip the same number of heads as tails, as the number of flips increases?. There are a number of other relevant questions on site. $\endgroup$– Glen_bCommented Jul 27, 2016 at 8:33
-
$\begingroup$ The confusion is about joint probability of the sequence vs. conditional probability of the final toss. For independent events you have $p[x_{1:11}=H]=p[x_{1:10}=H]p[x_{11}=H]$. When making the final toss the history is known ($p[x_{1:10}=H]=1$), so there is only a single unknown toss ... and $n=1$ is certainly not a large number. $\endgroup$– GeoMatt22Commented Dec 27, 2016 at 22:47
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Don't worry, this is a common misunderstanding. Let's consider this.
If I flip a fair coin 10 times, and all of them land on heads, according to the Law of Large Numbers I am more likely to get tails on the next flip. However, this is clearly the Gambler's Fallacy.
How am I misinterpreting the Law of Large Numbers?
The part I bolded is where the issue is, since the follow-up - that this is clearly the Gambler's Fallacy - is the correct angle.
The Law of Large Numbers states that
the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed.
The first issue is kind of a footnote - that ten trials isn't near enough to qualify for the "large number." (How large the large number has to be depends on the distribution - in the case of the binomial coin toss, it's roughly 30.)
There is a deeper issue, though - even if you got the coin to land on heads one million times in a row, the odds of the next toss are still 50/50. How do we reconcile this?
The Law of Large Numbers' stock-in-trade is in ratios. Over huge numbers, the ratio of "interesting events::possible outcomes" approximates the probability of an interesting event happening on any single occasion. For a concrete example - it's actually plausible that you flip a coin one million times as heads more than tails in the long run, where your ratio is something like 100.1:100 million (heads:tails). The "Big Idea" here is that, when you run a huge number of trials, that large number in the denominator will dilute the freak ten-heads streaks that appear and otherwise substantial numbers (say, 1,000,000) on either side of the scale matter a lot less to the ratio between them.
(For further reading, Jordan Ellenberg does a fantastic job explaining this early in his book, How Not to be Wrong.)
-
$\begingroup$ Where did you get the info that 30 is a large number for binomial distributions? $\endgroup$ Commented Jul 28, 2016 at 2:24
-
$\begingroup$ Whoops - I was thinking Central Limit Theorem. Both that point and the actual answer (n = sigma/(epsilon*delta^2) for a probability delta of being off by more than epsilon) are found here: stat.cmu.edu/~cshalizi/36-220/lecture-10.pdf $\endgroup$ Commented Jul 28, 2016 at 13:25