# Transition probabilities - Markov chains

I have a homogeneous Markov chain with transition matrix

I want to compute $P(Y_1 = 1| Y_2=2)$ where $Y_t, t=1,2$ is the observation at time $t$ and $Y_0=3$. I tried with Bayes' rule, so

$$P(Y_1 = 1| Y_2=2)= \frac{P(Y_2=2|Y_1=1)*P(Y_1=1)}{P(Y_2=2)}.$$

But from the transition matrix I can get only $P(Y_2=2|Y_1=1)=0.3$. How do we compute marginal probabilities $P(Y_2=2)$ and $P(Y_1=1)$ from a transition matrix?

• I wouldn't have used Bayes' rule to solve this; I would have computed $P(Y_1 = 1| Y_2=2)$ times $P(Y_1=1)$, given the fact that at $t=0$ I am in state 1. Then divide that for $P(Y_2=2|Y_0=1)$ computed using Chapman-Kolmogorov equation, I mean summing for all the possible $i$ intermediate states. – PhDing Jun 5 '16 at 19:57
• This link MIGHT help, scroll down and you will see a Marginal Distribution formula: stats.ox.ac.uk/~teh/teaching/dtc2014/Markov.pdf – EhsanF Jun 20 '16 at 15:28

I wouldn't have used Bayes' rule to solve this.

I would have computed $P(Y_1 = 1| Y_2=2)$ times $P(Y_1=1|Y_0=3)$. Then divide that for $P(Y_2=2|Y_0=3)$ computed using Chapman-Kolmogorov equation, I mean summing for all the possible $i$ intermediate states.

Remark: Bayes'rule is computed as $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$

• Ok, but I don't understand the point. I have $P(Y_1 = 1| Y_2=2)*P(Y_1=1|Y_0=1)$ divided by $P(Y_2=2|Y_0=1)$ if I could equate this quantity to something I could get the conditional probability – Bux Jun 5 '16 at 20:55
• Is this following fact true? $P(Y_2 = 2| Y_1=1, Y_0=3)= \frac{P(Y_1=1|Y_2=2, Y_0=3)*P(Y_2=2|Y_0=3)}{P(Y_1=1|Y_0=3)}$ – Bux Jun 5 '16 at 21:24
• Why are you assuming $Y_0=1$? (Indeed, in a comment to another answer the OP has stipulated that $Y_0=3$. That creates a divide-by-zero problem for your approach.) – whuber Jun 5 '16 at 21:24
• We are assuming $Y_0 =3$ – Bux Jun 5 '16 at 21:26
• Sorry about that, I made a mistake – PhDing Jun 6 '16 at 10:41

You would typically also be given a distribution $\pi$ over initial states. You could then read $P(Y_1 = x)$ directly off of $\pi$, and compute $P(Y_2 = x) = \sum _z P(Y_1 = z, Y_2 = x)$.

• I only know that at time 0 $Y_0=3$ – Bux Jun 5 '16 at 13:59
• OK, so now you can apply the transition matrix to find $\Pr(Y_1=1)$. What does that tell you? – whuber Jun 5 '16 at 15:56