Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Two friends, A and B, are tossing a fair coin together. Each of the first three coin tosses yields a "head", and they're about to toss it for a fourth time. A says the probability that the next coin toss yields a head is $0.5$. B says that since the probability that four consecutive coin tosses all turn up heads is $.0625$, the probability that the next toss yields a head should be $0.0625$ given the prior three heads.


I know intuitively that B is wrong and A is right, but can someone provide a mathematical explanation of why? Also, does Bayes' Theorem have any relevance here?

share|improve this question
2  
2  
Your later insertion of "fair" in the first line changes how this problem can be addressed and raises additional questions concerning how you know (or come to believe) the coin is fair. By asserting it is fair you anticipate an irrefutable simple answer: person2 cannot be correct, because by definition "fair" means the chance of heads is $1/2$ at every toss. Omitting the fairness assumption permits more creative solutions, such as the Bayes solution or an inference-based solution in which the question asks us how best to estimate the coin's properties based on the first three outcomes. –  whuber Jun 16 at 21:34
2  
By person2's logic, the probability of the coin coming up tails is also 0.0625 because that's the chance of getting the sequence HHHT. –  blahdiblah Jun 17 at 0:26
3  
There's no need to be insulting, @user1883050, we have an courtesy standard here, which you may want to review. As for the edits, you can roll them back if you like, they're intended to make the post clearer & more streamlined. If you click the "edited ___ ago" above my name, you can see the revision history & you'll see that the only edit I made was to remove the superfluous "thanks". It's nice, but our policy is not to add such. However, I did accept edits that were made by a lower rep user; I did (& continue to) find them an improvement. –  gung Jun 18 at 16:23
3  
@gung I'm sorry you're getting flak for trying to help--please accept my thanks, at least, for all you have been doing for us. I have deleted the comment in question. –  whuber Jun 18 at 18:06

6 Answers 6

up vote 4 down vote accepted

Person 2 seems to be confusing conditional probability of the next toss with the joint probability of the sequence. Denote the results of the $k$th coin toss by event $H_k$ that occurs iff the result is heads (otherwise, $\neg H_k$) Fairness of the coin is mathematically modeled by the following assumptions:

  1. $\mathbb{P}(H_k) = 0.5$ for all $k$.
  2. The events $H_1,H_2,\ldots$ are mutually independent.

As Avraham points out in his answer, these are assumptions. Whether these assumptions form a reasonable model of the real coin tossing process is a physical rather than mathematical question.

Person 2

Person 2 is computing the total probability that all 4 tosses are heads, \begin{equation} \mathbb{P}(H_1\cap H_2 \cap H_3 \cap H_4) \end{equation} applying the independence assumption, this equals \begin{equation} =\mathbb{P}(H_1)\mathbb{P}(H_2)\mathbb{P}(H_3)\mathbb{P}(H_4) \end{equation} and applying the first assumption, \begin{equation} = 0.5^4 = 0.0625. \end{equation} So, person 2 correctly computes the probability of obtaining a sequence of 4 heads. Unfortunately, that is not what they want to know.

Person 1

Person 1 understands that the question is about the probability of the next toss coming out heads, given the history of previous tosses \begin{equation} \mathbb{P}(H_4 \mid H_3\cap H_2\cap H_1) \end{equation} but, due to the independence assumption, this conditioning on the other events does not change the probability of $H_4$, thus the condition may simply be omitted: \begin{equation} =\mathbb{P}(H_4) = 0.5. \end{equation}

Thinking about the whole sequence

Person 2 was thinking about the sequence of all 4 tosses, while the question was worded in terms of the 4th toss. However, instead of considering the event of the next toss, person 1 could as well have computed the conditional probability of the whole sequence, conditional on what has already been observed \begin{equation} \mathbb{P}(H_4\cap H_3\cap H_2\cap H_1 \mid H_3 \cap H_2 \cap H_1) \end{equation} Here, one could apply Bayes' theorem or the definition of conditional probability: \begin{equation} =\frac{\mathbb{P}((H_4\cap H_3\cap H_2\cap H_1) \cap (H_3 \cap H_2 \cap H_1 ))}{\mathbb{P}(H_3 \cap H_2 \cap H_1)} = \frac{\mathbb{P}(H_4 \cap H_3 \cap H_2 \cap H_1)}{\mathbb{P}(H_3 \cap H_2 \cap H_1)} \end{equation} based on independence, this equals \begin{equation} =\frac{\mathbb{P}(H_4)\mathbb{P}(H_3)\mathbb{P}(H_2)\mathbb{P}(H_1)}{\mathbb{P}(H_3)\mathbb{P}(H_2)\mathbb{P}(H_1)} = \mathbb{P}(H_4) = 0.5. \end{equation} This second computation is unnecessarily complicated, as the computation 'Person 1' already gave the correct probability of the next toss. However, I think this could help convincing person 2 that one does not need to take the total sequence of 4 heads into account.

share|improve this answer
    
Nicely explained! –  Avraham Jun 16 at 19:54

Well, it depends. Firstly, is the coin fair? Secondly, is each flip independent of all other flips?

Most of the time this problem is posed, the assumption is that the coin is fair (Bernoulli, p= 0.5) and the flips are iid (independent, identically distributed). If so, the proper response is that the probability of a the next flip being heads is 0.5.

The "problem" with person B's assumption is that s/he is assuming that the four flips are not independent, and that prior flips have some way of "communicating themselves" to the last flip. This is not the case in the real world (we think 8-) ), and so person 2 is probably incorrect.

A Bayesian would look at this and say, three flips in a row heads? How certain are you that the coin is fair? To be fair (pun intended), three flips doth not a large sample make, but the Bayesian would start with the prior distribution that the coin is fair, take into account that three flips in a row were heads, and arrive at a new posterior for the coin for the fourth flip.

Update

Just for fun, the Bayesian would probably use the beta-binomial conjugate family here, starting with a Beta(1, 1) prior to represent uniformity. If so, given $y$ successes (heads) out of $n$ observations, the mean posterior distribution would be: $$ E(\theta\vert y) = \frac{\alpha + y}{\alpha + \beta + n} $$

So a Bayesian using this prior/likelihood assumption would say that there is a whopping 80% chance that the next flip is a heads.

Update 2

It is easiest to do the math using the beta/binomial convention. In general, the likelihood of the data under the assumption it is binomially distributed with parameter $\theta$ is: $$ p(y\vert\theta) \propto \theta^a\left(1 - \theta\right)^b $$

Where a and b are the instances of success and failure respectively. Now, the prior distribution of the parameter $\theta$, which represents the "success" rate (or probability of heads) will be assumed beta. Once again conveniently being able to ignore the constants that do not depend on the parameter itself, this can be written as: $$ p(\theta) \propto \theta^{\alpha - 1}\left(1 - \theta\right)^{\beta - 1} $$ So, calling $y$ the number of successes and $n$ the total observations (so failures = $n - yy$), we can write the posterior as proportional to: $$ \begin{align} p(\theta\vert y) &\propto p(y\vert\theta)p(\theta)\\ p(\theta\vert y) &\propto \theta^y(1 - \theta)^{n-y}\theta^{\alpha - 1}(1-\theta)^{\beta-1}\\ &\propto \theta^{y + \alpha - 1}(1 - \theta)^{n-y+\beta-1} \end{align} $$ Well, that last is just a beta with parameters $\alpha +y, \beta + n - y$.

Here, we started with a very special beta, with $\alpha = \beta = 1$, which is actually the uniform distribution on (0, 1). We also had $y = n = 3$, so we are left with a beta(4, 1), and given the mean of the beta is $\frac{\alpha}{\alpha + \beta}$ we have a mean expectation for $\theta$, which is the probability of heads after three flips, of $\frac{4}{5}$ or 80\%.

Of course, three flips isn't a good sample size from which to make far-reaching conclusions, but it helps.

share|improve this answer
    
Thanks. I added a clarification that the coin is fair. Would you please clarify the math involved in how the Bayesian would "take into account that three flips in a row were heads"? What would the prior calculate to? And the other components of the Bayesian equation? (I know I'm asking a lot, but I'd really appreciate it if you could take the time 8-) . Thanks again.). –  user1883050 Jun 16 at 19:27
    
Person 2's approach does not even yield probabilities that sum to 1 (3 heads and 1 tails is also 0.0625), thus it is not explainable by any dependency between the flips. –  Juho Kokkala Jun 16 at 19:29
    
The update is good--but you need to indicate what values $\alpha$ and $\beta$ have. It would be even clearer if you just replaced them by ones. –  whuber Jun 16 at 19:39
    
Was doing so as you commented, Dr. Huber, thanks. –  Avraham Jun 16 at 19:51

Person 2 is not wrong in thinking about the entire four-coin sequence. But Person 2 is failing to take into account that the first three tosses have already happened, that the results are already known to be heads, and that the probability of each of those first three tosses coming up heads is $1.0$, not $0.5$.

Before tossing any coins, the probability of getting a sequence of four heads in a row is $(0.5)^4=0.0625$.

If you have already tossed three heads, the probability of getting a sequence of four heads in a row is $1 \cdot 1 \cdot 1 \cdot 0.5 = 0.5$, exactly the same as the probability of getting heads on a single toss.

share|improve this answer

It is all in the conditioning (the sample space). Conditioned on 3 heads, the probability of a fourth is $0.5$.

And more generally, we write $P(A)$ but the sample space $\Omega$ is always there: $P(A)=P(A|\Omega)$. In your case $P(HHHH|HHH\cap\Omega)=P(H|\Omega)=0.5 \neq P(HHHH|\Omega)=0.5^4 $.

share|improve this answer

Your intuition is correct (If the tossing is fair etc., in real world six attempt returning the same result is a valid reason for doubting that because the chance is quite small but still imaginable).

The reasoning should be found in the inclusion or exclusion of previous attempts. As an outside observer you can take into account all attempts and calculate the change that 7 tosses yield the same side of the coin: .5 ^ 7.

Since all previous attempts are history and irrelevant for the upcoming changes the current change is .5.

Conclusion: if he says 'The chance of tossing the same side 4 times in a row is 1 - .5 ^4' he is right, but the first part has already happened. The change that he would reach this point is '1 - .5 ^ 3'. His next throw will have the 'normal' chance of .5

share|improve this answer

Okay, all the stats phDs and grad students are weighing in, in a most interesting but still probably overly-complicated matter to the average layperson.

The problem is you are confounding a sequence of events (the odds of getting 4 heads in a row) with just the next event (one heads).

The probability of getting 4 heads in a row is 0.0625 or 1/16.

The probability of getting 1 heads GIVEN you already got 3 heads in a row is 1/2, because the event is independent of all past events.

Here's a more intuitive way to think about it.

What are the odds of the sun coming up tomorrow? Approximately 100%, or 1.

What are the odds of you winning the lottery? Let's say 1 in one million.

What are the odds of you winning the lottery AND the sun rising tomorrow? 1 * 1/million = 1/million.

Now you just happened to win the lottery. What are the odds that the sun will rise tomorrow? They are still 1. They are not 1/million to average out to the sequential probability you established earlier.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.