infinite coin toss probability

Question

Suppose you have a fair coin. You start with $1$ dollar, if toss H, your money doubles, if toss T, your money halves. What is the expected value of the money you have if you toss the coin infinitely?

Why do the following two arguments lead to different answers? Why is 2 incorrect?

1. Let $X$ denote a toss, then $\mathbb{E}(X)=\frac{1}{2}2+\frac{1}{2}\frac{1}{2}=\frac{5}{4}$. Then We have $\mathbb{E}\left(\prod_{i=1}^nX_i\right)=\prod_{i=1}^n\mathbb{E}(X_i)=\left(\frac{5}{4}\right)^n$.

2. Let $M_n$ be the amount of money you have at step $n$, then $M_n=2^{n_H-n_T}$, with $n_H+n_T=n$, where $n_H$ is the number of heads in the first $n$ tosses, and $n_T$ is the number of tails in the first $n$ tosses. Since the coin is fair, so for large $n$, $n_H=n_T$, and hence $M_n\to1$.

Answer 1

For a fair coin $n_H/n_T\to 1$, but it is not the case that $n_H-n_T\to 0$. We know by the central limit theorem that the approximate distribution of $n_H$ is $N(n/2, n/4)$, so that the approximate distribution of $n_H-n_T$ is $N(0, n)$. That is, $n_H-n_T$ has mean zero, but has typical size $\sqrt{n}$.

The approximate distribution of $M_n=2^{n_H-n_T}$ is logNormal. If $$\log_2 M_n\sim N(0,n)$$ then $$\log M_n\sim N(0,(n)(\log 2)^2)$$

The mean of a logNormal distribution with log mean of $\mu$ and log variance of $\sigma^2$ is $\exp(\mu+\sigma^2/2)$, which comes to $$\exp(0+(n/2)(\log 2)^2)\approx 1.27^n$$

This isn't exactly the $(5/4)^n$ that you get from the first approach, but it's not that far off given the relatively crudeness of the Normal approximation for small $n$.

Answer 2

Your 1st case

You could make your expression $\mathbb{E}(X)=\frac{1}{2}2+\frac{1}{2}\frac{1}{2}=\frac{5}{4}$ more correctly like:

$$\mathbb{E}(M_{n+1})=\frac{1}{2}2\mathbb{E}(M_{n})+\frac{1}{2}\frac{1}{2}\mathbb{E}(M_{n})=\frac{5}{4} \mathbb{E}(M_{n})$$

and as a result:

$$\mathbb{E}(M_{n})= \left( \frac{5}{4} \right)^n$$

I believe that this is the correct expression as I can relate it to two other paths:

• computational By simulating a lot of cases I find that the power-law with a factor 5/4 seems correct.

  

   # settings
   set.seed(1)
   ktoss <- 1:50
   n <- 10^6
   Q <- rep(0,length(ktoss))
  
   # compute for 1 to 50 tosses   n <- 10^6 million trials
   for (k in ktoss) {
     t <- rbinom(n,k,0.5)
     Q[k-min(ktoss)+1] <- mean(2^(k-2*t))
   }  
  
   # plotting
   plot(ktoss,Q, log = "y")
   lines(ktoss,(5/4)^ktoss)
  

• Exact expression There is a direct way to compute the expectation for the power of a binomial distribution using the moment generating function (This is demonstrated in this question: Mean and variance of log-binomial distribution)

  For the binomial distribution we have

  $$E(e^{kX}) = M_{X}(k) = (1-p+p e^k)^n $$

  and using shifting and scaling properties of the moment generating function you can get to

  $$E(2^{2X_n-n}) = M_{2X_n-n}(\ln(2)) = e^{-ln(2)n}(1-p+p e^{ln(2) 2})^n = 1.25^n$$

---

I am not sure why the log-normal distribution does not approach the log-binomial distribution.

---

Your 2nd cse

The problem with the second argument is that

$$E(2^{X_n}) \neq 2^{E(X_n)}$$

But the expectation for the logarithm of the money is constant in time.

Answer 3

Some caution is warranted here: Starting with a 'fortune' of \$1, the variance of the fortune under the model described becomes infinite. The analyses for a finite number of coin tosses are OK, but the original question asks for behavior as $n \rightarrow \infty.$

It doesn't take a run of hugely many successive H's or T's to overflow or underflow a double precision representation of the fortune. (More money than there is in the world, or an infinitesimal fraction of a cent.) Eventually, in a long sequence of tosses, such long runs are inevitable.

For example, plots of fortunes for six simulated runs of the experiment (each through a planned 100,000 coin tosses) are shown below. The vertical scale for money is a log scale, so the horizontal line at $0$ represents no gain or loss.

set.seed(2020);  n = 10^5
par(mfrow=c(2,3))
 for(i in 1:6){
  ht = sample(c(.5,2), n, rep=T)
  money = cumprod(ht)
  plot(1:n, log(money), ylim=c(-300,300), type="l")
   abline(h=0, col="darkgreen")  }
par(mfrow=c(1,1))