Stuck on Penney's game question

Question

Suppose we have a game that assumes we have a fair coin. Two people who are playing can pick two patterns of coin flips. The players then flip a large coin a number of times and record each of the individual flips.

After the flips, they pay attention to the record of flips and decide which pattern came first in the sequence.

Using simulations, we can find:

1. HHT comes before THH about 1/4th of the time
2. THH comes before TTH about 1/3rd of the time
3. TTH comes before HTT about 1/4th of the time
4. HTT comes before HHT about 1/3rd of the time

Can someone explain to me why this is and what the intuition behind this game is?

Answer 1

Appropriate diagrams make these results intuitive -- and they can even lead to exact calculations with little effort.

---

Probability flows like water (or any other conserved substance). After observing a sequence $\omega$ of coin tosses, its probability is split into two "flow channels" according to the chance of heads, $p,$ and the chance of tails (which must be $1-p$ to conserve probability).

In technical notation, this picture describes two conditional probabilities $p = \Pr(\omega H\mid \omega)$ and $1-p = \Pr(\omega T\mid \omega).$

Because all examples in the question involve sequences of three flips, nothing interesting happens until after the first three flips. At this point there are $2^3=8$ possible states, such as THH (tails, then heads, then heads), and because the coin is fair, all states have a chance of $1/8.$

From then on, by focusing on the most recent run of three outcomes, each flip induces a transition from the current state to another state. For instance, a THH followed by a tails transitions to HHH.

The initial state probabilities are like pools of water, with a volume of $1/8$ in each pool. The transitions make the water flow out of each pool into two pools.

Here is the full diagram of all transitions. Solid red arrows are the transitions occurring when a head is observed; dashed blue arrows are the transitions occurring when a tail is observed.

By means of examples, I will show how to compute with this graph.

Example 1: THH vs. HHT

The game ends upon encountering either THH or HHT. We can model this by removing all outflows from these pools. Having done this, the water eventually will reach one of them and collect there.

The chance that THH is reached first equals the amount of water (probability) that accumulates in the THH basin.

Here is the preceding transition diagram with (i) the outflows from THH and HHT removed and (ii) THH outlined in black, HHT outlined in green.

Removing the outflows has disconnected the flow system. It is visually obvious that all water (probability) originally at HHH or HHT eventually ends in HHT. Therefore the chance HHT is reached before THH is $(2)\times(1/8) = 1/4.$ This is the first example of the question.

Example 2: THH vs. THT

Not all analyses are so simple. Here is the pruned graph for THH vs. THT.

Some pools flow eventually into both the terminal basins. However, there are obvious groups of pools: HHH eventually flows only into HHT. We might as well think of this as a pool with $1/8+1/8=1/4$ of the water (probability). HTT and TTT eventually flow into TTH. We might as well merge them into a pool with $3/8$ of the water. In doing these merges, we must retain all inflows and outflows. Here is the resulting "reduced graph."

A neat way to compute how much water (probability) ends in the THH basin is to create a "flow fractions" graph. It begins by assigning 100% of the water in THH to the THH basin and 0% of the water in THT to the THH basin. Then it works backwards through the graph. For instance, HTH splits its outflows between THH and THT. Thus, only 50% (1/2) of the water in HTH winds up in THH. The same is true for the combined HTT/TTT/TTH pool: half its water (probability) ends up in THH. Finally, the outflow from the HHH/HHT pool is split between HTH and the HTT/TTT/TTH pools. Since each of the latter contribute half their water to THH, the same must be true for HHH/HHT.

Here is the resulting flow fractions graph, drawn to parallel the reduced graph:

Finally, recall how much water begins in each pool: $1/8$ in THH, $1/8$ in HTH, $1/8$ in THT, $3/8$ in HTT/TTT/TTH, and $2/8$ in HHH/HHT. Multiplying these amounts by the flow fractions gives

$$\frac{1}{8} \times 1 + \frac{1}{8} \times \frac{1}{2} + \frac{1}{8}\times 0 + \frac{3}{8}\times \frac{1}{2} + \frac{2}{8} \times \frac{1}{2} = \frac{1}{2}.$$

Thus, the chance THH is encountered first is $1/2.$

Example 3: THH vs. TTH

This is the second example of the question.

Again begin by pruning the graph to remove all outfalls from the target basins:

Collect groups of interconnected pools into the reduced graph.

It is already obvious that TTH has the advantage, because most of the pools drain wholly or partly into it.

The challenge this time is to compute the flow fractions: the water can swirl endlessly between HTH and THT. However, half is lost forever to THH when flowing out of HTH and half is lost forever to TTH when flowing out of THT (because eventually it flows only into TTH). This HTH-THT loop, though, seems to make it impossible to backtrack through the graph.

The solution is to leave one of the fractions as an unknown $x.$ Here, I have arbitrarily chosen to let $x$ be the flow fraction for THT.

The flow fraction for HTH is $1/2\times 1$ because of the flow into THH plus $1/2\times x$ because of the flow into THT.

Upon visiting all the nodes, we obtain two expressions for the THT flow fraction. This yields the equation

$$x = \frac{1}{2}\times \frac{1+x}{2} + \frac{1}{2}\times 0,$$

whose (unique) solution is $x=1/3.$ Now we can just read off the remaining flow fractions: $(1+x)/2 = 2/3$ for HTH and $1/3$ for the HHH/HHT pool. The chance of reaching THH first therefore is

$$\frac{1}{8}\times 1 + \frac{1}{8}\times \frac{2}{3} + \frac{1}{8}\times \frac{1}{3} + \frac{3}{8}\times 0 + \frac{2}{8} \times \frac{1}{3} = \frac{1}{3}.$$

Comments

These graph calculations generalize. It is clear how they can be applied to comparing any two terminal states in this game. But no change needs to be made when terminating the game at multiple states. For instance, suppose you play this game against an opponent who wins if either TTH or THT is reached and you win if THH is first reached. What are your chances of winning?

Here are the pruned graph, the reduced graph, and the flow fractions. You can do the arithmetic.

Answer 2

This type of problem relates to "string-counts", which in turn relate to deterministic finite automata (DFAs) for strings (see this related question). The simplest way to understand these problems is to construct a DFA for the "language" of interest in the problem (i.e., the set of strings being counted). In the figure below I show the minimised DFA for your first case, with heads represented by the blue arrows and tails represented by the red arrows. There are seven different states in the minimised DFA, and the final states of interest are at the bottom of the figure. As you can see from the picture, the transitions in the DFA are not symmetric with respect to these final states.

As you can see from the figure, there are a substantial number of transitions into the state T in this DFA, and this is the main reason why the the state THH tends to occur before the state HHT. The first string is a more "forgiving" string on the first run (i.e., starting from the null state); once you are in the state T you are guaranteed to reach the state THH before the state HHT. If you would like some more insight into this, consider the number of transitions into these two states from the NULL state, noting that each transition of head/tails is equiprobable under your model.

---

Implementation in R: Below I implement the first of these string-count problems in my own custom DFA function in R (to be released at a later date). This output shows the transition table for the DFA (matching the arrows in the figure) and shows the transition probability matrix and steady state for the DFA. As you can see, the two outcomes of interest to you will occur with equal probability in the steady state, and will therefore tend to occur in roughly the same proportions in the long run. However, the state THH tends to occur before the state HHT due to the large number of transitions into the early components of that string.

Deterministic Finite Automaton (DFA) 

DFA for a language containing 2 strings 

---------------------Transitions and final states---------------------- 
 
         | H T | F[1] F[2]
State[0] | 2 1 | 0    0   
State[1] | 3 1 | 0    0   
State[2] | 4 1 | 0    0   
State[3] | 5 1 | 0    0   
State[4] | 4 6 | 0    0   
State[5] | 4 6 | 0    1   
State[6] | 3 1 | 1    0   

-------------------------Symbol probabilities-------------------------- 

              H   T
Probability 0.5 0.5

--------------------Stationary state probabilities--------------------- 
 
           State[0] State[1] State[2] State[3] State[4] State[5] State[6]
Stationary        0    0.375        0     0.25    0.125    0.125    0.125

---------------------Transition probability matrix--------------------- 

         State[0] State[1] State[2] State[3] State[4] State[5] State[6]
State[0]        0      0.5      0.5      0.0      0.0      0.0      0.0
State[1]        0      0.5      0.0      0.5      0.0      0.0      0.0
State[2]        0      0.5      0.0      0.0      0.5      0.0      0.0
State[3]        0      0.5      0.0      0.0      0.0      0.5      0.0
State[4]        0      0.0      0.0      0.0      0.5      0.0      0.5
State[5]        0      0.0      0.0      0.0      0.5      0.0      0.5
State[6]        0      0.5      0.0      0.5      0.0      0.0      0.0

-------------------Alphabet and language information------------------- 

The alphabet contains 2 symbols 
 H T 
 
String[1] contains 3 symbols 
 H H T 

String[2] contains 3 symbols 
 T H H 

-----------------------------------------------------------------------