5
$\begingroup$

First, my probability game is pretty elementary and many of the probability formulas still look pretty intimidating.

That said, I am trying to understand how to calculate the probability of a run of $x$ within a larger set of total tosses.

Specifically, what is the probability of getting $7$ heads in a row in a total of $100$ tosses.

A formula that I am looking at, but not sure if it is correct, is as follows:

$\Pr = 1-(1-0.5^7)^{(100-7+1)} = .522$

Does the above formula accurately capture the likelihood of the event occurring?

$\endgroup$
2
  • $\begingroup$ I think your answer is here. $\endgroup$ Commented Mar 1, 2017 at 18:21
  • $\begingroup$ Antoni, thank you. This should probably be obvious to me, but much of that formula I get based on the surrounding text except for what "T" represents. Can you help with that bit, please? $\endgroup$ Commented Mar 1, 2017 at 19:10

2 Answers 2

3
$\begingroup$

First the empirical proof to motivate the long explanation:

We can get the answer with a one-line Monte Carlo simulation, corresponding to $1$ million trials, which in R would be made simple thanks to rle() and the suggestion in this answer:

> set.seed(0)
> mean(replicate(1e6, TRUE %in% (with(rle(sample(c(0,1),100,T)), lengths[values==1]) >= 7)))
> [1] 0.317231

Now the closed, algebraic solution:

I am going to assume lack of familiarity with Markov chains, and make it very easy to follow.

The idea is that we go through $100$ steps, and at each transition, there is a given probability distribution for things to stay as they are or to change. Here by "things" I simply mean the number of consecutive heads ($\text H$'s).

So we start tossing the coin and we have the following distribution of events:

$$\small\begin{bmatrix}s_0=\text{T}&s_1=\text{H}_1\\1/2&1/2\end{bmatrix}$$

either tails or heads, and it is a fair coin... The key is to realize that if it was tails, we are back to were we began - let's call it state $0$ or $s_0.$ Otherwise, we are taking the first step towards building a sequence of $7$ consecutive heads, i.e. $s_1.$

We toss again. And this is now really critical to appreciate... If we had actually built up the beginning of a run (i.e. if we had a head in the first toss) we would be in the blue position right before the second toss:

$$\small\begin{bmatrix}s_0&\color{blue}{s_1}\\1/2&\color{blue}{1/2}\end{bmatrix}$$

and we would have, naturally, a probability of $1/2$ to build up a longer run (a run of $2$ consecutive $\text{H}$'s, or $s_2$) by simply getting another head, or we could go back to the beginning by getting a $\text{T}$. We have no interest in runs that are less than $7$ as per your initial question.

So we can either obtain a head and keep building up our sequence of $7$ heads, or go back to $s_0$. If we are lucky we land in $s_2$ - a "cumulative" run of $2$ heads, which would look like this:

$$\small\begin{bmatrix}\quad&s_0&\color{blue}{s_1}&\color{green}{s_2}\\\text{1st toss}&1/2&\color{blue}{1/2}&0\\\text{2nd toss} &1/2&0&\color{green}{1/2}\end{bmatrix}$$

Now we are either back to "square one" or $s_0$, with nothing to account for, if a tail came up on the second toss, or we have advanced to $s_2$, and each scenario has $1/2$ probabilities. Notice that there is no scenario in which we could find ourselves in $s_1$: we either built up a sequence of two consecutive heads, i.e. $s_2=\{\text{H,H}\}$ if we got a head, or we are back to zero.

Next round...

$$\small \begin{bmatrix}\quad&s_0&\color{blue}{s_1}&\color{green}{s_2}&\color{red}{s_3}\\\text{1st toss}&1/2&\color{blue}{1/2}&0&0\\\text{2nd toss} &1/2&0&\color{green}{1/2}&0\\\text{3rd toss}&1/2&0&0&\color{red}{1/2}\end{bmatrix}$$

That's right... We either have a run of $3$ heads or we are back to zero. So we keep on going and we get...

$$\small \begin{bmatrix}\quad&s_0&\color{blue}{s_1}&\color{green}{s_2}&\color{red}{s_3}&\cdots&\color{orange}{s_7}\\\text{1st toss}&1/2&\color{blue}{1/2}&0&0&\cdots&0\\\text{2nd toss} &1/2&0&\color{green}{1/2}&0&\cdots&0\\\text{3rd toss}&1/2&0&0&\color{red}{1/2}&\cdots&0\\\vdots&\vdots&\vdots&\vdots&\vdots&\cdots&\vdots\\ \text{7th toss}&1/2&0&0&0&\cdots&\color{orange}{1/2}\end{bmatrix}$$

But we are not done, because we are going to multiply this transition matrix $T$ over and over again $100$ times by itself - i.e. we are going to $T^{100}$ (more on this below). And we have to prevent that the ongoing probability tally of $s_7$ in the last column gets modified by itself. How do we do it? We add a row of all zeros except for a $1$ in the last entry as below. Why? Because it will consistently guarantee that the last column, which is what we really want (our "accounting column" or the probabilities of $s_7$), does not have any influence from one step to the next in the recalculation - only the columns to the left of $s_7$ will influence or contribute to updating $s_7$ at every pass:

$$\small T=\begin{bmatrix}\quad&s_0&\color{blue}{s_1}&\color{green}{s_2}&\color{red}{s_3}&\cdots&\color{orange}{s_7}\\\text{1st toss}&1/2&\color{blue}{1/2}&0&0&\cdots&0\\\text{2nd toss} &1/2&0&\color{green}{1/2}&0&\cdots&0\\\text{3rd toss}&1/2&0&0&\color{red}{1/2}&\cdots&0\\\vdots&\vdots&\vdots&\vdots&\vdots&\cdots&\vdots\\ \text{7th toss}&1/2&0&0&0&\cdots&\color{orange}{1/2}\\\quad&0&0&0&0&\cdots&\color{purple}1\end{bmatrix}$$

This is the transition matrix or stochastic matrix, $T$. Check absorbing states here.

In multiplying this matrix times itself, you can observe that the last entry of each row will be multiplied by $0$ (except for the last row we just added), leaving the last, column $\color{orange}{s_7}$ completely nullified in the $[\text{row}]\cdot[\text{column}]$ dot product.

Now for the home stretch... We have built this matrix as a story, with the implicit logic that we went from $s_1$ to $s_2$ as we transitioned from the one-element, $s_1$, to the two-element run, $s_2$ (or went back to $s_0$) in the second toss with certain probabilities. However, we never considered the probability of transitioning to $s_1$ on the fourth toss, for example, which is going to depend on the probabilities of all possible prior states. The probability distribution at each point in time will depend on the prior distribution, explaining why we have to raise $T$ to the $100$ power. Let's hone down on one of these multiple dot products within the powers of $T$: we focus on the evolution of probabilities in the first row and see that when we dot it with the third column in the process of calculating $T^2$

$$\small \begin{bmatrix}s_0&\color{blue}{s_1}&\color{green}{s_2}&\color{red}{s_3}&\cdots&\color{orange}{s_7}\\1/2&\underbrace{\;\color{blue}{1/2}\;}_{\text{step } t}&0&0&\cdots&0\\\end{bmatrix}\cdot\tiny\begin{bmatrix}\color{green}{s_2}\\0\\\text{step }t+1:\;\color{green}{1/2}\\0\\0\\0\\0\end{bmatrix}$$

we split (multiply by $\color{green}{1/2}$) the probability associated with a run in the immediate prior state ($t=s_1$) (at this point $\color{blue}{1/2}$) to assign a probability to a run that is one element longer ($t+1=s_2).$

We can find the final result after calculating $T^{100}$ in exactly this position:

$$\small T^{100}=\begin{bmatrix}\quad&s_0&\color{blue}{s_1}&\color{green}{s_2}&\color{red}{s_3}&\cdots&\color{orange}{s_7}\\&&&&&&\bbox[5px,border:2px solid red]{\text{answer}}\\&&&&&&&\\&&&&&&&\\&\vdots&\vdots&\vdots&\vdots&\cdots&\vdots\\ &&&&&&&\\&0&0&0&0&\cdots&\color{purple}1\end{bmatrix}$$

after performing $T^{100}$.

Notice how in every calculation step, we dot the first row with each column, in what is really a bunch of and and or statements to calculate the probability of each state in the first row.

This ends up being $\bbox[5px,border:2px solid red]{T^{100}[1,8]= 0.31752}$.

Calculated in R:

require(expm)
m= matrix(c( .5,  .5,0,0,0,0,0,0,
             .5, 0, .5,0,0,0,0,0,
             .5, 0,0, .5,0,0,0,0,
             .5, 0,0,0, .5,0,0,0,
             .5, 0,0,0,0, .5,0,0,
             .5, 0,0,0,0,0, .5,0,
             .5, 0,0,0,0,0,0, .5,
              0, 0,0,0,0,0,0,  1), nrow = 8, byrow=T)
t(c(1,0,0,0,0,0,0,0)) %*% (m%^%100 %*% c(0,0,0,0,0,0,0,1))
$\endgroup$
4
  • $\begingroup$ This is really interesting and very helpful! Thank you! $\endgroup$ Commented Mar 2, 2017 at 14:06
  • $\begingroup$ Thanks for this great derivation! When you just want a numeric probability for a concrete set of parameters, just type streak of 7 successes in 100 trials in WolframAlpha. $\endgroup$
    – pommy
    Commented Jun 27, 2022 at 7:55
  • $\begingroup$ This is great! One tiny comment: the original post asked about runs of heads not runs of both heads and tails. It seems your MC code would somehow need to incorporate $values==1 as well as $lengths==7. Not knowing R, I'm not sure how to do this in a one-liner. E.g., when I check a run of 2 in 3 tosses, I expect 0.25 because there are 2 sets of rolls: 011 and 110 that have a run of 2 heads. But an MC one-liner mean(replicate(1e6, sum(rle(sample(c(0,1), 3, T))$lengths ==2) > 0)) yields ~0.5 because rle considers 100 and 001 as runs of two too, even though those are runs of tails. $\endgroup$
    – Andrew
    Commented Aug 18, 2022 at 15:39
  • $\begingroup$ @Andrew Thank you very much for pointing this out. I am happy now we the solution in my recent edit to the answer. I use the rle() code to jus focus on streaks of ones. $\endgroup$ Commented Aug 21, 2022 at 20:47
2
$\begingroup$

This question is a variant of the "hot hands" phenomenon, much studied wrt shot sequences in basketball and, more generally, "luck vs skill" questions. The standard assumption is that the calculation of the probability of a sequence of 7 heads in a row is an unconditional estimate across all 1,000 tosses. Using this approach, the conclusion is that the "hot hand" phenomenon does not exist, which has had significant importance for decision-making.

However, and as Andrew Gelman notes in a blog post from July 2015 titled Hey-guess what? There really is a hot hand! (1), this unconditional approach is incorrect. As documented in a paper referenced by Gelman (2), a better approach is a conditional estimate where, given a head, the probability of the next toss (or shot) also being a head initializes a new sequence of conditional estimates. Under this conditional method, the hot hand phenomenon is demonstrable.

Like you, I'm not an expert with probabilistic reasoning explained with Latex formulas. So I will leave the derivations and proofs to the experts writing the paper referenced by Gelman. It's quite readable.

(1) http://andrewgelman.com/2015/07/09/hey-guess-what-there-really-is-a-hot-hand/

(2) Miller and Sanjuro, Surprised by the Gambler's and Hot Hand Fallacies? A Truth in the Law of Small Numbers, https://papers.ssrn.com/sol3/papers2.cfm?abstract_id=2627354

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.