Statistics concept to explain why you're less likely to flip the same number of heads as tails, as the number of flips increases?

Question

I'm working on learning probability and statistics by reading a few books and writing some code, and while simulating coin flips I noticed something that struck me as slightly counter to one's naive intuition. If you flip a fair coin $n$ times, the ratio of heads to tails converges towards 1 as $n$ increases, exactly as you would expect. But on the other hand, as $n$ increases, it appears that you become less likely to flip the exact same number of heads as tails, thereby getting a ratio of exactly 1.

For example (some output from my program)

For 100 flips, it took 27 experiments until we got an exact match (50 HEADS, 50 TAILS)
For 500 flips, it took 27 experiments until we got an exact match (250 HEADS, 250 TAILS)
For 1000 flips, it took 11 experiments until we got an exact match (500 HEADS, 500 TAILS)
For 5000 flips, it took 31 experiments until we got an exact match (2500 HEADS, 2500 TAILS)
For 10000 flips, it took 38 experiments until we got an exact match (5000 HEADS, 5000 TAILS)
For 20000 flips, it took 69 experiments until we got an exact match (10000 HEADS, 10000 TAILS)
For 80000 flips, it took 5 experiments until we got an exact match (40000 HEADS, 40000 TAILS)
For 100000 flips, it took 86 experiments until we got an exact match (50000 HEADS, 50000 TAILS)
For 200000 flips, it took 96 experiments until we got an exact match (100000 HEADS, 100000 TAILS)
For 500000 flips, it took 637 experiments until we got an exact match (250000 HEADS, 250000 TAILS)
For 1000000 flips, it took 3009 experiments until we got an exact match (500000 HEADS, 500000 TAILS)

My question is this: is there a concept / principle in statistics / probability theory that explains this? If so, what principle / concept is it?

Link to code if anyone is interested in seeing how I generated this.

-- edit --

For what it's worth, here's how I was explaining this to myself earlier. If you flip a fair coin $\mathtt n$ times and count the number of heads, you're basically generating a random number. Likewise if you do the same thing and count the tails, you're also generating a random number. So if you count both, you're really generating two random numbers, and as $\mathtt n$ gets larger, the random numbers are getting larger. And the larger the random numbers you generate, the more chances there are for them to "miss" each other. What makes this interesting is that the two numbers are actually linked in a sense, with their ratio converging towards one as they get bigger, even though each number is random in isolation. Maybe it's just me, but I find that sort of neat.

Answer 1

Note that the case where the number of heads and the number of tails are equal is the same as "exactly half the time you get heads". So let's stick to counting the number of heads to see if it's half the number of tosses or equivalently comparing the proportion of heads with 0.5.

The more you flip, the larger the number of possible counts of heads you can have -- the distribution becomes more spread out (e.g. an interval for the number of heads containing 95% of the probability will grow wider as the number of tosses increases), so the probability of exactly half heads will tend to go down as we toss more.

Correspondingly, the proportion of heads will take more possible values; see here, where we move from 100 tosses to 200 tosses:

With 100 tosses we can observe a proportion of 0.49 heads or 0.50 heads or 0.51 heads (and so on -- but nothing in between those values), but with 200 tosses, we can observe 0.49 or 0.495 or 0.50 or 0.505 or 0.510 - the probability has more values to "cover" and so each will tend to get a smaller share.

Consider than you have $2n$ tosses with some probability $p_i$ of getting $i$ heads (we know these probabilities but it's not critical for this part), and you add two more tosses. In $2n$ tosses, $n$ heads is the most likely outcome ($p_n>p_{n\pm 1}$ and it goes down from there).

What's the chance of having $n+1$ heads in $2n+2$ tosses?

(Label these probabilities with $q$ so we don't confuse them with the previous ones; also let P(HH) be the probability of "Head,Head" in the next two tosses, and so on)

$q_{n+1} = p_{n-1} P(HH) + p_n (P(HT)+P(TH)) + p_{n+1} P(TT)$

$\qquad < p_{n} P(HH) + p_n (P(HT)+P(TH)) + p_{n} P(TT) = p_n$

i.e. if you add two more coin-tosses, the probability of the middle value naturally goes down because it averages the most likely (middle) value with the average of the smaller values either side)

So as long as you're comfortable that the peak will be in the middle (for $2n= 2,4,6,...$), the probability of exactly half heads must decrease as $n$ goes up.

---

In fact we can show that for large $n$, $p_n$ decreases proportionally with $\frac{1}{\sqrt{n}}$ (unsurprisingly, since the distribution of the standardized number of heads approaches normality and the variance of the proportion of heads decreases with $n$).

---

As requested, here's R code that produces something close to the above plot:

 x1 = 25:75
 x2 = 50:150
 plot(x1 / 100, dbinom(x1, 100, 0.5), type = "h",
       main = "Proportion of heads in 100 and 200 tosses",
       xlab = "Proportion of heads",
       ylab = "probability")
 points(x2 / 200, dbinom(x2, 200, 0.5), type = "h", col = 3)

Answer 2

Well we know that the Law of Large Numbers is what is guaranteeing the first conclusion of your experiement, namely, that if you flip a fair coin $n$ times, the ratio of heads to tails converges towards 1 as $n$ increases.

So no problems there. However, that about all the Law of Large Numbers tells us in this scenario.

But so now, think about this problem more intuitively. Think about flipping a coin a small number of times, for example: $n=2,4,8,10$.

When you flip a coin twice, i.e. $n=2$, think of the possible scenarios of the two flips. (Here $H$ will denote heads and $T$ will denote tails). On the fist flip you could have gotten $H$ and on the second flip you could have gotten $T$. But that's just one way the two flips could have come up. You could have also gotten on the first flip $T$ and on the second flip $H$, and all other possible combinations. So at the end of the day, when you flip 2 coins, the possible combinations you could see on the two flips are $$S=\{HH,HT,TH,TT\}$$ and so there are 4 possible scenarios for flipping $n=2$ coins.

If you were to flip 4 coins then the possible number of combinations you could see would be $$S=\{HHHH,HHHT,HHTH,HTHH,THHH,HHTT,HTTH,TTHH,THHT,THTH,HTHT,HTTT,THTT,TTHT,TTTH,TTTT\}$$ and so there are 16 possible scenarios for flipping $n=4$ coins.

Flipping $n=8$ coins leads to 256 combinations.

Flipping $n=10$ coins leads to 1,024 combinations.

And in particular, flipping any number $n$ coins leads to $2^n$ possible combinations.

Now, let's try and approach this problem probabilistic point of view. Looking back at the case when $n=2$, we know that the probability of getting exactly the same number of Heads and Tails (i.e., as you put it a ratio of exactly 1) is $$Pr(\text{Ratio of exactly 1})=\frac{2}{4}=0.5$$ When $n=4$, we know that the probability of getting exactly the same number of Heads and Tails is $$Pr(\text{Ratio of exactly 1})=\frac{6}{16}=0.375$$

And in general, as $n$ tends to grow larger we have that that the probability of getting exactly the same number of Heads and Tails goes to 0.

In other words, as $n\rightarrow\infty$, we have that $$Pr(\text{Ratio of exactly 1})\rightarrow 0$$

And so, to answer your question. Really what you are observing is just a consequence of the fact that there will be far more combinations of coin flips where the number of heads and tails are not equal as compared to the number of combinations where they are equal.

---

---

As @Mark L. Stone suggests, if you are comfortable with the binomial formula and binomial random variables, then you can use that to show the same argument.

Let $X$ be the number of heads recorded when flipping a fair coin $n$ times. we can regard $X$ as a random variable coming from a binomial distribution, i.e., $X\sim Bin(n,p=0.5)$ (here we assume $p=0.5$ because we are dealing with a fair coin) then the probability of getting exactly the same number of heads as the number of tails (i.e., a ratio of exactly 1) is

$$Pr(\text{Ratio of exactly 1})=Pr\left(X=\frac{n}{2}\right)= {n \choose n/2} 0.5^{n/2}(0.5)^{n-n/2}={n \choose n/2} 0.5^{n}$$

Now, once again, as $n$ tends to grow large, the above expression tends towards 0 because ${n \choose n/2}0.5^n\rightarrow 0$ as $n\rightarrow\infty$.

Answer 3

See Pascal's Triangle.

The likelihood of coin flip outcomes is represented by the numbers along the bottom row. The outcome of equal heads and tails is the middle number. As the tree grows larger (i.e., more flips), the middle number becomes a smaller proportion of the sum of the bottom row.

Answer 4

Maybe it helps to outline that this is related to the arcsine law. It says that for one path of outcomes the probability that the path stays for most the time in the positive or negative domain is much higher than that it is going up and down than you expect from intuition. Here some links:

http://www.math.unl.edu/~sdunbar1/ProbabilityTheory/Lessons/BernoulliTrials/ExcessHeads/excessheads.shtml

https://en.wikipedia.org/wiki/Arcsine_law

Answer 5

While the ratio of heads to tails converges to 1, the range of possible numbers becomes wider. (I'm making the numbers up). Say for 100 throws the probability is 90% that you have between 45% and 55% heads. That's 90% that you get 45 to 55 heads. 11 possibilities for the number of heads. About 9% roughly that you get equal numbers of heads and tails.

Say for 10,000 throws the probability is 95% that you get between 49% and 51% heads. So the ratio has come a lot closer to 1. But now you have between 4,900 and 5,100 heads. 201 possibilities. Chance of equal numbers is only roughly about 0.5%.

And with a million throws you are quite sure to have between 49.9% and 50.1% heads. That's a range from 499,000 to 501,000 heads. 2,001 possibilities. The chance is now down to 0.05%.

Ok, the maths was made up. But this should give you an idea about the "why". Even though the ratio comes closer to 1, the number of possibilities becomes greater, so that hitting exactly half head, half tails, becomes less and less likely.

Another practical effect: It is unlikely in practice that you have a coin where the probability of throwing heads is exactly 50%. It might be 49.99371% if you have a really good coin. For small number of throws this doesn't make a difference. For large numbers, the percentage of heads will converge to 49.99371%, not 50%. If the number of throws is large enough, throwing 50% or more heads will become very, very unlikely.

Answer 6

Well, one thing to note is that with an even number of flips (otherwise the probability of equal heads and tails flips is of course exactly zero), the most probable outcome will always be the one with exactly as many heads flips as tails flips.

The distribution of $n$ flips is given by the coefficients of the polynomial $$\bigl(\frac{1+x}2\bigr)^n\qquad .$$ So for even $n$, the probability is $$p_n = 2^{-n}{n \choose n/2}\qquad .$$

Using Stirling's approximation for $n!$, you arrive at something like $$p \approx\frac1{\sqrt{\pi n/2}}$$ for the probability of exactly $n/2$ heads (and correspondingly tails) flips for $n$ overall flips. So the absolute probability of this outcome converges to 0 but much slower than most of the other outcomes, with the extreme cases of 0 heads (or alternatively 0 tails) flips being $2^{-n}$.

Answer 7

Suppose you flip a coin twice. There are four possible outcomes: HH, HT, TH, and TT. In two of these, you have an equal number of heads and tails, so there's a 50% chance that you get the same number of heads and tails.

Now suppose you flip a coin 4,306,492,102 times. Do you expect a 50 percent chance that you'll wind up with exactly 2,153,246,051 heads and 2,153,246,051 tails?