Fair coin tosses simulation and Law of Large Numbers

Question

My question is related with a simulation Professor John Guttag did in the 15th lecture of the course MIT Opencourseware "Introduction to C.S." The program simulates multiple fair coin tosses experiments. It sets multiple values for n, the number of tosses in each experiment and for each n, it repeats the experiment m times to infer some values (m is fixed for every n).

The number of tosses n is set to be powers of two, from $2^{k_{min}}$ to $2^{k{max}}$ so n is growing as the simulation progress.

For each n, the experiment is performed m times. In each of the m times, the number of Heads $H$ and Tails $T = n - H$ is stored and then the average (over the m repetitions) of $\frac{H}{T}$ and $|H -T|$ is computed, so we have an $\frac{H}{T}$ and $|H-T|$ average for each n.

For me it is very surprising that Guttang considers quite obvious that as n grows the fraction $\frac{H}{T}$ converges to 1 and that $|H-T|$ grows kind of linear with n.

I consider this far from obvious if you try to prove it mathematically.

Using random variables, by definition $H$ is a Binomail random variable $H \text{~} Bin(n,\frac{1}{2})$

So by the Law of Large Numbers we know that the average computed over the $m$ repetitions tends to the expected value of $H$, $E[H] = np = \frac{n}{2}$ as m tends to infinity. I think that Prof Guttag assumes it is obvious that the ratio $\frac{H}{T}$ tends to one because $H$ tends to $\frac{n}{2}$ and so does $T$ (which is correct) and this seems to imply that $\frac{H}{T}$ tends to $\frac{\frac{n}{2}}{\frac{n}{2}} = 1$

This reasoning for me is not correct. In general $E[\frac{H}{T}] \neq \frac{E[H]}{E[T]}$. In this case in fact, $E[\frac{H}{T}] = E[\frac{H}{n-H}] = \infty$ because the denominator can be zero with a probability equal $(\frac{1}{2})^n$. Is this correct or am I missing something?

Also, I think it is more "obvious" to think that $|H-T|$ tends to zero instead of thinking that grows with n. In fact, if m is large enough, $E[H-T] = E[2H - N] = 0 ~~\forall n$. I know that $E[|H-T|] \gt E[H-T]$ but not easy to see that it has to grow with n, any proof about this?

I would appreciate any comment if I am wrong about anything or if you have seen the Prof Gutag lecture and I have not understood it correctly.

Lecture 15th URL: https://www.youtube.com/watch?v=VqZBqoZgL7k&index=21&list=PLB2BE3D6CA77BB8F7

Minute 15th show the graphs about the computed averages.

Answer 1

You may consider this slightly different way of framing it easier to deal with:

1. The difference in the proportion of heads and the proportion of tails indeed converges to 0.

2. $|H-T| = |H-(n-H)| = |2H-n| = 2|H-\frac{n}{2}|$. Even though $E(H)=\frac{n}{2}$, the typical distance of $H$ from its mean does not shrink and $n$ grows larger; in fact the distribution of $H$ becomes more spread out. For $n=2m$, the mean deviation of a binomial is $m{2m\choose m} 2^{-2m}$, which for large $m$ approaches $\sqrt{\frac{m}{\pi}}=\sqrt{\frac{n}{2\pi}}$.

   At $n=100$, the exact mean deviation is $50\cdot {100\choose 50} 2^{-100}\approx 3.98$ while the large-$n$ approximation is $\sqrt{\frac{100}{2\pi}}\approx 3.99$.

   For large $n$ the mean deviation is $\sqrt{\frac{2}{\pi}}$ ($\approx 0.8$) of the standard deviation; it's often easier to work from that since most people remember the standard deviation of a binomial and the scaling factor isn't hard to remember (or derive).

(I'll try to come back to the issue of $\frac{H}{T}$ in the original as soon as time permits)