For a Markov Chain is $𝑃(𝑞_𝑡|𝑞_{𝑡+1},…,𝑞_𝑇)$ equals to $𝑃(𝑞_𝑡| 𝑞_{𝑡+1})$?

Question

I am new to Markov Chains and using this concept in statistics.

For a Markov Chain, may I say that $𝑃(𝑞_𝑡|𝑞_{𝑡+1},…,𝑞_𝑇)$ equals to $𝑃(𝑞_𝑡| 𝑞_{𝑡+1})$?

If yes, how can I prove that?

Answer 1

The Markov property applies both forwards and backwards

The Markov property holds that $p(q_{t+k}|q_t,...,q_{t+k-1}) = p(q_{t+k}|q_{t+k-1})$ for all $t \in \mathbb{Z}$ and $k \in \mathbb{N}^+$. This property is usually stated as a forward equation (i.e., the probability density of an outcome conditional on past values), but it also implies the result in your question, which is the same essential property as a backward equation (i.e., the probability density of an outcome conditional on future values). This is shown as follows:

$$\begin{equation} \begin{aligned} p(q_t| q_{t+1},...,q_T) &\overset{q_t}{\propto} p(q_t, q_{t+1},...,q_T) \\[12pt] &= p(q_t) \prod_{i=1}^{T-t} p(q_{t+i}| q_{t},...,q_{t+i-1}) \\[6pt] &= p(q_t) \prod_{i=1}^{T-t} p(q_{t+i}| q_{t+i-1}) \\[6pt] &\overset{q_t}{\propto} p(q_t) p(q_{t+1}|q_t) \\[12pt] &= p(q_t,q_{t+1}) \\[12pt] &\overset{q_t}{\propto} p(q_t|q_{t+1}). \\[6pt] \end{aligned} \end{equation}$$

(The third step in this working uses the forward equation for the Markov property.) So, the take-away message is that the Markov property applies both forward and backward --- one implies the other.

Answer 2

Ben correctly points out that a Markov chain is Markovian both forwards and backwards (+1). This is always true, but it is not the same thing as reversibility. In particular, reversibility requires the existence of a stationary distribution, call it $\pi$.

For a chain with a stationary distribution (aka a marginal distribution that doesn't change depending on what time point you're at), $$ p(q_{t-1} \mid q_t) = \frac{p(q_t \mid q_{t-1})\pi(q_{t-1})}{\pi(q_t)} \tag{1}. $$

This is the definition of a reversible Markov chain. If you multiply both sides of the above by the denominator on the right hand side, you will get the more familiar definition of reversibility: $$ p(q_{t-1} \mid q_t)\pi(q_t) = p(q_t \mid q_{t-1})\pi(q_{t-1}) $$ which says being in state $q_t$ and then $q_{t-1}$ a moment later has the same chances as being in state $q_{t-1}$ and then state $q_t$ a moment later.

You'll notice that sub-scripting elements of the state space with a time index isn't great for expressing this idea very well.