# Can we calculate the theoretical stationary distribution from a continuous Markov chain?

I have the transition distribution $$p(X_{t+1}|X_t=x_t) = \text{N}(\phi x_t,1)$$ where $$−1<\phi<1$$.

Can we calculate the stationary distribution and its mean and variance? I know I can do that if the Markov chain is discrete, by multiplying the transition probability matrix $$Q$$ many times. How can I do that if the transition probability is continuous?

I am talking in general not just for this example. Can we calculate the theoretical stationary distribution of a continuous Markov chain if we have the transition distribution?

Your Markov chain is a Gaussian $$\text{AR}(1)$$ process with unit error variance:

$$X_{t+1} = \phi X_t + \varepsilon_{t+1} \quad \quad \quad \varepsilon_t \sim \text{IID N}(0,1).$$

The elements in this process can be written as weighted sums of normal error terms (in $$\text{MA}(\infty)$$ form), so the stationary distribution (if it exists) will be a normal distribution, $$X_t \sim \text{N}(\mu, \sigma^2)$$.

It remains only to find a mean and variance parameter that yield a stationary distribution. To do this, we apply the recursive equation to give equations for the mean and variance parameters. Under the condition of stationarity we have a fixed mean, which means gives the mean equation:

$$\mu = \mathbb{E}(X_{t+1}) = \mathbb{E}(\phi X_t + \varepsilon_{t+1}) = \phi \mathbb{E}(X_t) + \mathbb{E}(\varepsilon_{t+1}) = \phi \mu.$$

Since $$-1 < \phi < 1$$ this yields the unique solution $$\mu = 0$$. Under the condition of stationarity we also have a fixed variance, which means gives the variance equation:

$$\sigma^2 = \mathbb{V}(X_{t+1}) = \mathbb{V}(\phi X_t + \varepsilon_{t+1}) = \phi^2 \mathbb{V}(X_t) + \mathbb{V}(\varepsilon_{t+1}) = 1 + \phi^2 \sigma^2.$$

This equation yields the solution $$\sigma^2 = 1/(1-\phi^2)$$. Thus, the stationary distribution exists, and it is:

$$X_t \sim \text{N} \Bigg( 0, \frac{1}{1-\phi^2} \Bigg).$$

Note that this form is well-known as the stationary distribution of an $$\text{AR}(1)$$ process with unit variance on the noise terms. It can easily be generalised to broader $$\text{AR}(1)$$ processes with a non-zero mean, or non-unit variance for the error term.

• Thank you so much for the answer but when you multiply the stationary distribution from your answer with the transition distribution from my question, It won't equal the posterior. I know that In Markov chain, $\pi = Q* \pi$ where $\pi$ is the posterior distribution and $Q$ is the transition distribution. Am I right? Aug 23, 2019 at 17:33
• Sorry, just noticed this comment - I've answered it at your other question.
– Ben
Sep 1, 2019 at 3:43
• Could you take a look at my related question on MSE: math.stackexchange.com/q/4728844/47771? Jul 2 at 13:25