0
$\begingroup$

I've been searching around for a bit and I can't find out if these kinds of operations are allowed as it's kind of exploiting the Markov property.

Consider a Markov chain belonging to a the state space $S = \{1,2,3\}$ with the goal of finding (where $a,b,c \in S$): $$\mathbb{P}(X_1=a, X_2=b, X_3=c | X_0 = a)$$

Given that I'm not too sure on how to solve the joint probability left of the conditional probability I re-wrote it using the property: $$\mathbb{P}(A|B) = \frac{\mathbb{P}(A,B)}{\mathbb{P}(B)}$$

Which results in: $$ \frac{ \mathbb{P}(X_1=a, X_2=b, X_3=c, X_0 = a) }{ \mathbb{P}(X_0 = a)} \quad \quad ,\left(\mathbb{P}(X_0=a) \neq 0\right) $$

Then by applying the chain rule to the numerator (and thus cancelling the term in the denominator): $$ \mathbb{P}(X_1=a | X_2=b, X_3=c, X_0=a) \cdot \mathbb{P}(X_2=b | X_3=c, X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a) $$

Then by the Markov property, this becomes: $$ \mathbb{P}(X_1=a | X_0=a) \cdot \mathbb{P}(X_2=b | X_0=a) \cdot \mathbb{P}(X_3=c | X_0=a) $$

Which I can then solve.


However, note that: $$ \mathbb{P}(X,Y) = \mathbb{P}(Y,X) $$

So tracking back a couple of steps before the chain rule was applied we could re-write the fraction as:

$$ \frac{ \mathbb{P}(X_3=c, X_2=b, X_1=a, X_0 = a) }{ \mathbb{P}(X_0 = a)} \quad \quad ,\left(\mathbb{P}(X_0=a) \neq 0\right) $$

Then by applying the chain rule as done previously: $$ \mathbb{P}(X_3=c | X_2=b, X_1=a, X_0=a) \cdot \mathbb{P}(X_2=b | X_1=a, X_0=a) \cdot \mathbb{P}(X_1=a | X_0=a) $$

Similarly, by the Markov property this becomes: $$ \mathbb{P}(X_3=c | X_2=b) \cdot \mathbb{P}(X_2=b | X_1=a) \cdot \mathbb{P}(X_1=a | X_0=a) $$

As with before, I can now solve this (and much easier than the final equation in the first section).


Which is where I am very unsure as to if that kind of operation is allowed, as far as I am aware... or are they equivalent?

Many thanks in advance.

$\endgroup$
0
$\begingroup$

For the equation in the first section, I don't think you can use the Markov property to simplify P(X1=a|X2=b,X3=c,X0=a) to P(X1=a|X0=a) because P(X2) is dependent on P(X1). I think the second one looks ok.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.