2
$\begingroup$

I am new to Markov Chains and using this concept in statistics.

For a Markov Chain, may I say that $𝑃(𝑞_𝑡|𝑞_{𝑡+1},…,𝑞_𝑇)$ equals to $𝑃(𝑞_𝑡| 𝑞_{𝑡+1})$?

If yes, how can I prove that?

$\endgroup$
4
$\begingroup$

The Markov property applies both forwards and backwards

The Markov property holds that $p(q_{t+k}|q_t,...,q_{t+k-1}) = p(q_{t+k}|q_{t+k-1})$ for all $t \in \mathbb{Z}$ and $k \in \mathbb{N}^+$. This property is usually stated in its backward sense (i.e., the probability density of an outcome conditional on past values), but it also implies the result in your question, which is the same essential property in the forward sense (i.e., the probability density of an outcome conditional on future values). This is shown as follows:

$$\begin{equation} \begin{aligned} p(q_t| q_{t+1},...,q_T) &\overset{q_t}{\propto} p(q_t, q_{t+1},...,q_T) \\[12pt] &= p(q_t) \prod_{i=1}^{T-t} p(q_{t+i}| q_{t},...,q_{t+i-1}) \\[6pt] &= p(q_t) \prod_{i=1}^{T-t} p(q_{t+i}| q_{t+i-1}) \\[6pt] &\overset{q_t}{\propto} p(q_t) p(q_{t+1}|q_t) \\[12pt] &= p(q_t,q_{t+1}) \\[12pt] &\overset{q_t}{\propto} p(q_t|q_{t+1}). \\[6pt] \end{aligned} \end{equation}$$

(The third step in this working uses the backwards version of the Markov property.) So, the take-away message is that the Markov property applies both forward and backward --- one implies the other.

$\endgroup$
0
$\begingroup$

Ben correctly points out that a Markov chain is Markovian both forwards and backwards (+1). This is always true, but it is not the same thing as reversibility. In particular, reversibility requires the existence of a stationary distribution, call it $\pi$.

For a chain with a stationary distribution (aka a marginal distribution that doesn't change depending on what time point you're at), $$ p(q_{t-1} \mid q_t) = \frac{p(q_t \mid q_{t-1})\pi(q_{t-1})}{\pi(q_t)} \tag{1}. $$

This is the definition of a reversible Markov chain. If you multiply both sides of the above by the denominator on the right hand side, you will get the more familiar definition of reversibility: $$ p(q_{t-1} \mid q_t)\pi(q_t) = p(q_t \mid q_{t-1})\pi(q_{t-1}) $$ which says being in state $q_t$ and then $q_{t-1}$ a moment later has the same chances as being in state $q_{t-1}$ and then state $q_t$ a moment later.

You'll notice that sub-scripting elements of the state space with a time index isn't great for expressing this idea very well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.