2
$\begingroup$

If the Markov chain converged then $$\pi = Q* \pi$$where $ \pi$ is the posterior distribution and $Q$ is the transition distribution(it's a matrix in the discrete case).
I tried to apply that on the continuous case of Markov chain in this example where
the transition distribution is:

$$p(X_{t+1} | X_t=x_t) = \text{N}(\phi x_t, 1)$$ and the posterior(stationary) distribution is $$X_t \sim \text{N} \Bigg( 0, \frac{1}{1-\phi^2} \Bigg)$$

The product of them both doesn't equal the posterior.

$\endgroup$
  • $\begingroup$ An analog of the first line in the continuous setting is Kolmogorov's backward and forward equations. You can solve them for the stationary distribution and related metrics. In general, a friendly reference on the topic: Ross, S. M. (2009). Introduction to Probability Models (10th ed). Academic Press. $\endgroup$ – stans - Reinstate Monica Aug 24 '19 at 6:31
3
+50
$\begingroup$

That stationary distribution is correct. Using the law of total probability, you have:

$$\begin{equation} \begin{aligned} p(T_{t+1} = x) &= \int \limits_\mathbb{R} p(X_{t+1} = x | X_t = r) \cdot p(X_t = r) \ dr \\[6pt] &= \int \limits_{-\infty}^\infty \text{N}(x | \phi r, 1) \cdot \text{N} \bigg( r \bigg| 0, \frac{1}{1-\phi^2} \bigg) \ dr \\[6pt] &= \int \limits_{-\infty}^\infty \frac{1}{\sqrt{2 \pi}} \exp \bigg( -\frac{1}{2} (x - \phi r)^2 \bigg) \cdot \sqrt{\frac{1-\phi^2}{2 \pi}} \exp \bigg( -\frac{1}{2} (1-\phi^2) r^2 \bigg) \ dr \\[6pt] &= \frac{\sqrt{1-\phi^2}}{2 \pi} \int \limits_{-\infty}^\infty \exp \bigg( - \frac{1}{2} (x - \phi r)^2 - \frac{1}{2} (1-\phi^2) r^2 \bigg) \ dr \\[6pt] &= \frac{\sqrt{1-\phi^2}}{2 \pi} \int \limits_{-\infty}^\infty \exp \bigg( - \frac{1}{2} \bigg[ (x - \phi r)^2 + (1-\phi^2) r^2 \bigg] \bigg) \ dr \\[6pt] &= \frac{\sqrt{1-\phi^2}}{2 \pi} \int \limits_{-\infty}^\infty \exp \bigg( - \frac{1}{2} \bigg[ x^2 - 2 \phi x r + \phi^2 r^2 + r^2 - \phi^2 r^2 \bigg] \bigg) \ dr \\[6pt] &= \frac{\sqrt{1-\phi^2}}{2 \pi} \int \limits_{-\infty}^\infty \exp \bigg( - \frac{1}{2} \bigg[ x^2 - 2 \phi x r + r^2 \bigg] \bigg) \ dr \\[6pt] &= \frac{\sqrt{1-\phi^2}}{2 \pi} \int \limits_{-\infty}^\infty \exp \bigg( - \frac{1}{2} \bigg[ x^2 (1 - \phi^2) + (r-\phi x)^2 \bigg] \bigg) \ dr \\[6pt] &= \frac{\sqrt{1-\phi^2}}{\sqrt{2 \pi}} \exp \bigg( -\frac{1}{2} (1-\phi^2) x^2 \bigg) \int \limits_{-\infty}^\infty \frac{1}{\sqrt{2 \pi}} \exp \bigg( - \frac{1}{2} (r-\phi x)^2 \bigg) \ dr \\[6pt] &= \text{N} \bigg( x \bigg| 0, \frac{1}{1-\phi^2} \bigg) \times \int \limits_{-\infty}^\infty \text{N} (r|\phi x,1) \ dr \\[6pt] &= \text{N} \bigg( x \bigg| 0, \frac{1}{1-\phi^2} \bigg). \\[6pt] \end{aligned} \end{equation}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.