I've been searching around for a bit and I can't find out if these kinds of operations are allowed as it's kind of exploiting the Markov property.

Consider a Markov chain belonging to a the state space $S = \{1,2,3\}$ with the goal of finding (where $a,b,c \in S$): $$\mathbb{P}(X_1=a, X_2=b, X_3=c | X_0 = a)$$

Given that I'm not too sure on how to solve the joint probability left of the conditional probability I re-wrote it using the property: $$\mathbb{P}(A|B) = \frac{\mathbb{P}(A,B)}{\mathbb{P}(B)}$$

Which results in: $$ \frac{ \mathbb{P}(X_1=a, X_2=b, X_3=c, X_0 = a) }{ \mathbb{P}(X_0 = a)} \quad \quad ,\left(\mathbb{P}(X_0=a) \neq 0\right) $$

Then by applying the chain rule to the numerator (and thus cancelling the term in the denominator): $$ \mathbb{P}(X_1=a | X_2=b, X_3=c, X_0=a) \cdot \mathbb{P}(X_2=b | X_3=c, X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a) $$

Then by the Markov property, this becomes: $$ \mathbb{P}(X_1=a | X_0=a) \cdot \mathbb{P}(X_2=b | X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a) $$

Which I can then solve.

However, note that: $$ \mathbb{P}(X,Y) = \mathbb{P}(Y,X) $$

So tracking back a couple of steps before the chain rule was applied we could re-write the fraction as:

$$ \frac{ \mathbb{P}(X_3=c, X_2=b, X_1=a, X_0 = a) }{ \mathbb{P}(X_0 = a)} \quad \quad ,\left(\mathbb{P}(X_0=a) \neq 0\right) $$

Then by applying the chain rule as done previously: $$ \mathbb{P}(X_3=c | X_2=b, X_1=a, X_0=a) \cdot \mathbb{P}(X_2=b | X_1=a, X_0=a) \cdot \mathbb{P}(X_1=a | X_0=a) $$

Similarly, by the Markov property this becomes: $$ \mathbb{P}(X_3=c | X_2=b) \cdot \mathbb{P}(X_2=b | X_1=a) \cdot \mathbb{P}(X_1=a | X_0=a) $$

As with before, I can now solve this (and much easier than the final equation in the first section).

Which is where I am very unsure as to if that kind of operation is allowed, as far as I am aware... or are they equivalent?

Many thanks in advance.


2 Answers 2


Scalar multiplication is commutative, so:

$$ \mathbb{P}(X_1=a | X_0=a) \cdot \mathbb{P}(X_2=b | X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a),$$

is equal to:

$$ \mathbb{P}(X_3=c | X_2=b) \cdot \mathbb{P}(X_2=b | X_1=a) \cdot \mathbb{P}(X_1=a | X_0=a).$$

All you have done here is to derive this expression by two different (long-winded) methods. It is actually much simpler to derive this result from the multiplication rule for conditional probability, combined with the Markov property:

$$\begin{aligned} \mathbb{P}(X_1=a, X_2=b, X_3=c | X_0 = a) &= \mathbb{P}(X_3=c | X_0 = a, X_1=a, X_2=b ) \\[6pt] &\quad \times \mathbb{P}(X_2=b | X_0 = a, X_1=a) \\[6pt] &\quad \times \mathbb{P}(X_1=a | X_0 = a) \\[6pt] &= \mathbb{P}(X_3=c | X_2=b ) \cdot \mathbb{P}(X_2=b | X_1=a) \\[6pt] &\quad \times \mathbb{P}(X_1=a | X_0 = a) \\[6pt] \end{aligned}$$


For the equation in the first section, I don't think you can use the Markov property to simplify $P(X_1=a\mid X_2=b,X_3=c,X_0=a) $ to $ P(X_1=a\mid X_0=a) $ because $P(X_2)$ is dependent on $P(X_1). $ I think the second one looks okay.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.