4
$\begingroup$

I have the transition distribution $p(X_{t+1}|X_t=x_t) = \text{N}(\phi x_t,1)$ where $−1<\phi<1$.

Can we calculate the stationary distribution and its mean and variance? I know I can do that if the Markov chain is discrete, by multiplying the transition probability matrix $Q$ many times. How can I do that if the transition probability is continuous?

I am talking in general not just for this example. Can we calculate the theoretical stationary distribution of a continuous Markov chain if we have the transition distribution?

$\endgroup$

1 Answer 1

2
$\begingroup$

Your Markov chain is a Gaussian $\text{AR}(1)$ process with unit error variance:

$$X_{t+1} = \phi X_t + \varepsilon_{t+1} \quad \quad \quad \varepsilon_t \sim \text{IID N}(0,1).$$

The elements in this process can be written as weighted sums of normal error terms (in $\text{MA}(\infty)$ form), so the stationary distribution (if it exists) will be a normal distribution, $X_t \sim \text{N}(\mu, \sigma^2)$.

It remains only to find a mean and variance parameter that yield a stationary distribution. To do this, we apply the recursive equation to give equations for the mean and variance parameters. Under the condition of stationarity we have a fixed mean, which means gives the mean equation:

$$\mu = \mathbb{E}(X_{t+1}) = \mathbb{E}(\phi X_t + \varepsilon_{t+1}) = \phi \mathbb{E}(X_t) + \mathbb{E}(\varepsilon_{t+1}) = \phi \mu.$$

Since $-1 < \phi < 1$ this yields the unique solution $\mu = 0$. Under the condition of stationarity we also have a fixed variance, which means gives the variance equation:

$$\sigma^2 = \mathbb{V}(X_{t+1}) = \mathbb{V}(\phi X_t + \varepsilon_{t+1}) = \phi^2 \mathbb{V}(X_t) + \mathbb{V}(\varepsilon_{t+1}) = 1 + \phi^2 \sigma^2.$$

This equation yields the solution $\sigma^2 = 1/(1-\phi^2)$. Thus, the stationary distribution exists, and it is:

$$X_t \sim \text{N} \Bigg( 0, \frac{1}{1-\phi^2} \Bigg).$$

Note that this form is well-known as the stationary distribution of an $\text{AR}(1)$ process with unit variance on the noise terms. It can easily be generalised to broader $\text{AR}(1)$ processes with a non-zero mean, or non-unit variance for the error term.

$\endgroup$
3
  • $\begingroup$ Thank you so much for the answer but when you multiply the stationary distribution from your answer with the transition distribution from my question, It won't equal the posterior. I know that In Markov chain, $\pi = Q* \pi$ where $\pi$ is the posterior distribution and $Q$ is the transition distribution. Am I right? $\endgroup$
    – floyd
    Aug 23, 2019 at 17:33
  • $\begingroup$ Sorry, just noticed this comment - I've answered it at your other question. $\endgroup$
    – Ben
    Sep 1, 2019 at 3:43
  • $\begingroup$ Could you take a look at my related question on MSE: math.stackexchange.com/q/4728844/47771? $\endgroup$
    – 0xbadf00d
    Jul 2 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.