# Markov chain joint/conditional probability properties

I've been searching around for a bit and I can't find out if these kinds of operations are allowed as it's kind of exploiting the Markov property.

Consider a Markov chain belonging to a the state space $S = \{1,2,3\}$ with the goal of finding (where $a,b,c \in S$): $$\mathbb{P}(X_1=a, X_2=b, X_3=c | X_0 = a)$$

Given that I'm not too sure on how to solve the joint probability left of the conditional probability I re-wrote it using the property: $$\mathbb{P}(A|B) = \frac{\mathbb{P}(A,B)}{\mathbb{P}(B)}$$

Which results in: $$\frac{ \mathbb{P}(X_1=a, X_2=b, X_3=c, X_0 = a) }{ \mathbb{P}(X_0 = a)} \quad \quad ,\left(\mathbb{P}(X_0=a) \neq 0\right)$$

Then by applying the chain rule to the numerator (and thus cancelling the term in the denominator): $$\mathbb{P}(X_1=a | X_2=b, X_3=c, X_0=a) \cdot \mathbb{P}(X_2=b | X_3=c, X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a)$$

Then by the Markov property, this becomes: $$\mathbb{P}(X_1=a | X_0=a) \cdot \mathbb{P}(X_2=b | X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a)$$

Which I can then solve.

However, note that: $$\mathbb{P}(X,Y) = \mathbb{P}(Y,X)$$

So tracking back a couple of steps before the chain rule was applied we could re-write the fraction as:

$$\frac{ \mathbb{P}(X_3=c, X_2=b, X_1=a, X_0 = a) }{ \mathbb{P}(X_0 = a)} \quad \quad ,\left(\mathbb{P}(X_0=a) \neq 0\right)$$

Then by applying the chain rule as done previously: $$\mathbb{P}(X_3=c | X_2=b, X_1=a, X_0=a) \cdot \mathbb{P}(X_2=b | X_1=a, X_0=a) \cdot \mathbb{P}(X_1=a | X_0=a)$$

Similarly, by the Markov property this becomes: $$\mathbb{P}(X_3=c | X_2=b) \cdot \mathbb{P}(X_2=b | X_1=a) \cdot \mathbb{P}(X_1=a | X_0=a)$$

As with before, I can now solve this (and much easier than the final equation in the first section).

Which is where I am very unsure as to if that kind of operation is allowed, as far as I am aware... or are they equivalent?

Scalar multiplication is commutative, so:

$$\mathbb{P}(X_1=a | X_0=a) \cdot \mathbb{P}(X_2=b | X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a),$$

is equal to:

$$\mathbb{P}(X_3=c | X_2=b) \cdot \mathbb{P}(X_2=b | X_1=a) \cdot \mathbb{P}(X_1=a | X_0=a).$$

All you have done here is to derive this expression by two different (long-winded) methods. It is actually much simpler to derive this result from the multiplication rule for conditional probability, combined with the Markov property:

\begin{aligned} \mathbb{P}(X_1=a, X_2=b, X_3=c | X_0 = a) &= \mathbb{P}(X_3=c | X_0 = a, X_1=a, X_2=b ) \\[6pt] &\quad \times \mathbb{P}(X_2=b | X_0 = a, X_1=a) \\[6pt] &\quad \times \mathbb{P}(X_1=a | X_0 = a) \\[6pt] &= \mathbb{P}(X_3=c | X_2=b ) \cdot \mathbb{P}(X_2=b | X_1=a) \\[6pt] &\quad \times \mathbb{P}(X_1=a | X_0 = a) \\[6pt] \end{aligned}

For the equation in the first section, I don't think you can use the Markov property to simplify P(X1=a|X2=b,X3=c,X0=a) to P(X1=a|X0=a) because P(X2) is dependent on P(X1). I think the second one looks ok.