Biased coins and Markov processes

Question

Good day, I am attempting an optional exercise and I am finding it hard to interpret the problem in terms of matrices and vectors.

Coin 1 has probability 0.4 of coming up heads, and coin 2 has probability 0.8 of coming up heads. The coin to be flipped initially is equally likely to be coin 1 or coin 2. Thereafter, if the flipped coin shows heads then coin 1 is chosen for the next flip, and if the flipped coin shows tails then coin 2 is chosen for the next flip.

Let $X_0$ be the coin chosen for the initial flip, and, for $n \geq1$, let Xn be the coin chosen for the nth flip after the initial flip.

(a) Explain why $X_0,X_1,X_2, . . .$ is a Markov chain. Write down its statespace and its transition matrix.

(b) Let $p^{(n)}$ be the probability row vector giving the distribution of $X_n$. Find $p^{(0)}, p^{(1)}, p^{(2)}$.

(c) Write down the probability that coin 1 is chosen for the second flip after the initial flip.

(d) Find the probability that coin 1 is chosen for the second and third flip after the initial flip.

What I thought of is that the state space is $S=(1,2)$ and the transition matrix would be

$$\begin{pmatrix} 0.4 & 0.6\\\ 0.8 & 0.2\end{pmatrix}$$

After doing this, I think the rest of the questions will be easy to approach

Using this, I get that $p^{(0)}=( \frac{1}{2}, \frac{1}{2})$ as the initial toss has equal probability of being in state 1 or 2.

Using the answer below, I get that

$p^{(1)}=( 0.6, 0.4)$

$p^{(2)}=( 0.56, 0.44)$

The answer to $c)$ is then $0.6$.

But the last part ($d)$) is still puzzling me.

I feel like there is some Bayes involved and that the answer is not simply $0.6*0.56$.

Answer 1

I'm not sure where you got three states. There are two states that each $X_n$ can be in: coin $1$ or coin $2$. This suggests using the state space $S = \{1, 2\}$.

The process $(X_n)_{n=0}^\infty$ is a Markov chain because the coin being used on a particular flip only depends on the result of the previous coin flip, not on the entire history of coin tosses up to that point.

To compute the transition probabilities, we can first think intuitively in terms of the problem description. Suppose the coin in the $n$th round is coin $1$. The only way that the coin in the next round is coin $1$ is if the current coin shows heads. Since the current coin is coin $1$, this happens with probability $0.4$. To formalize this slightly, we just showed that

$$ \begin{aligned} P(X_{n+1} = 1 \mid X_n = 1) &= P(\text{The $(n+1)$th coin is coin $1$} \mid \text{The $n$th coin is coin $1$}) \\ &= P(\text{The $n$th coin shows heads} \mid \text{The $n$th coin is coin $1$}) \\ &= P(\text{Coin $1$ shows heads}) \\ &= 0.4. \end{aligned} $$ This is one of the four transition probabilities needed for the transition matrix. You can similarly compute the remaining three transition probabilities to get the transition matrix.

And like you said, once you have the transition matrix, the remaining parts are straightforward matrix computations (Note that $p^{(0)} = (1/2, 1/2)$ since you are told that the initial coin has an equal chance of being coin $1$ or coin $2$).

Answer 2

Find the probability that coin 1 is chosen for the second and third flip after the initial flip.

Now you just need to calculate the probability that the chosen coin is not just chosen in the second flip (which is 0.6 as answered in part c) but also chosen in the third flip.

Consider the probability that the coin 1 will be chosen in the third flip given that it has already been chosen in the 2nd flip.