Markov chain joint/conditional probability properties

Question

I've been searching around for a bit and I can't find out if these kinds of operations are allowed as it's kind of exploiting the Markov property.

Consider a Markov chain belonging to a the state space $S = \{1,2,3\}$ with the goal of finding (where $a,b,c \in S$): $$\mathbb{P}(X_1=a, X_2=b, X_3=c | X_0 = a)$$

Given that I'm not too sure on how to solve the joint probability left of the conditional probability I re-wrote it using the property: $$\mathbb{P}(A|B) = \frac{\mathbb{P}(A,B)}{\mathbb{P}(B)}$$

Which results in: $$ \frac{ \mathbb{P}(X_1=a, X_2=b, X_3=c, X_0 = a) }{ \mathbb{P}(X_0 = a)} \quad \quad ,\left(\mathbb{P}(X_0=a) \neq 0\right) $$

Then by applying the chain rule to the numerator (and thus cancelling the term in the denominator): $$ \mathbb{P}(X_1=a | X_2=b, X_3=c, X_0=a) \cdot \mathbb{P}(X_2=b | X_3=c, X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a) $$

Then by the Markov property, this becomes: $$ \mathbb{P}(X_1=a | X_0=a) \cdot \mathbb{P}(X_2=b | X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a) $$

Which I can then solve.

---

However, note that: $$ \mathbb{P}(X,Y) = \mathbb{P}(Y,X) $$

So tracking back a couple of steps before the chain rule was applied we could re-write the fraction as:

$$ \frac{ \mathbb{P}(X_3=c, X_2=b, X_1=a, X_0 = a) }{ \mathbb{P}(X_0 = a)} \quad \quad ,\left(\mathbb{P}(X_0=a) \neq 0\right) $$

Then by applying the chain rule as done previously: $$ \mathbb{P}(X_3=c | X_2=b, X_1=a, X_0=a) \cdot \mathbb{P}(X_2=b | X_1=a, X_0=a) \cdot \mathbb{P}(X_1=a | X_0=a) $$

Similarly, by the Markov property this becomes: $$ \mathbb{P}(X_3=c | X_2=b) \cdot \mathbb{P}(X_2=b | X_1=a) \cdot \mathbb{P}(X_1=a | X_0=a) $$

As with before, I can now solve this (and much easier than the final equation in the first section).

---

Which is where I am very unsure as to if that kind of operation is allowed, as far as I am aware... or are they equivalent?

Many thanks in advance.

Answer 1

Scalar multiplication is commutative, so:

$$ \mathbb{P}(X_1=a | X_0=a) \cdot \mathbb{P}(X_2=b | X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a),$$

is equal to:

$$ \mathbb{P}(X_3=c | X_2=b) \cdot \mathbb{P}(X_2=b | X_1=a) \cdot \mathbb{P}(X_1=a | X_0=a).$$

All you have done here is to derive this expression by two different (long-winded) methods. It is actually much simpler to derive this result from the multiplication rule for conditional probability, combined with the Markov property:

$$\begin{aligned} \mathbb{P}(X_1=a, X_2=b, X_3=c | X_0 = a) &= \mathbb{P}(X_3=c | X_0 = a, X_1=a, X_2=b ) \\[6pt] &\quad \times \mathbb{P}(X_2=b | X_0 = a, X_1=a) \\[6pt] &\quad \times \mathbb{P}(X_1=a | X_0 = a) \\[6pt] &= \mathbb{P}(X_3=c | X_2=b ) \cdot \mathbb{P}(X_2=b | X_1=a) \\[6pt] &\quad \times \mathbb{P}(X_1=a | X_0 = a) \\[6pt] \end{aligned}$$

Answer 2

For the equation in the first section, I don't think you can use the Markov property to simplify $P(X_1=a\mid X_2=b,X_3=c,X_0=a) $ to $ P(X_1=a\mid X_0=a) $ because $P(X_2)$ is dependent on $P(X_1). $ I think the second one looks okay.