# non-invertible weakly stationary AR processes

ok, so I know (for example) that $X_t = 2X_{t-1} + e_t$ for iid $e_t$ is stationary.

But how do I go about proving the condition $Cov(X_r, X_s) = Cov(X_{r+t}, X_{s+t})$ for all integer $r, s, t$ without having to resort to the transformation which requires $X_t$ to depend on the future rather than the past?

Edit: For clarification, the above example can be rewritten as:

$X_t = \frac{1}{2} X_{t+1} + u_t$, with $u_t$ some new white noise process. Now we can write down the infinite MA representation of this process, that is,

$X_t = \sum_{j=0}^{\infty} (\frac{1}{2})^{j+1} u_{t+j}$.

And from this, we can show that the series is stationary. But now, the series depends on the future rather than the past.

• I am not quite clear about your question. can you explain that more? Jul 2 '15 at 9:24

@Hamed has already pointed out to you that the process is not stationary (as you claim in the text of your question) nor weakly stationary (as in the "Subject" line of your posting). Rather than trying to understand his point, you put him down by claiming that he does not "grasp the concepts." A -1 for that.

From $X_t = 2X_{t-1} + e_t$, we can deduce that

\begin{align} X_{r+t} &= 2X_{r+t-1} + e_{r+t}\\ &= 2[2X_{r+t-2} +e_{r+t-1}] + e_{r+t}\\ &= 4X_{r+t-2} + 2e_{r+t-1} + e_{r+t}\\ &= 4[2X_{r+t-3} +e_{r+t-2}] + 2e_{r+t-1} + e_{r+t}\\ &= 8X_{r+t-3} + 4e_{r+t-2}+ 2e_{r+t-1} + e_{r+t}\\ &= \cdots\\ X_{r+t} &= 2^tX_r + \sum_{i=0}^{t-1} 2^{i}e_{r+t-i}\\ \text{Similarly}\qquad \qquad&\\ X_{s+t} &= 2^tX_s + \sum_{i=0}^{t-1} 2^{i}e_{s+t-i} \end{align} The bilinearity of covariance result $$\operatorname{cov}\left(\sum_i a_iX_i, \sum_j b_j Y_j\right) = \sum_i \sum_j a_i b_j \operatorname{cov}\left(X_i, Y_j\right)$$ shows that $$\operatorname{cov}\left(X_{r+t}, X_{s+t}\right) = 2^{2t}\operatorname{cov}\left(X_{r}, X_{s}\right) + \scriptstyle{\text{other gobbledygook that you can fill in}}$$ and so proving that $\operatorname{cov}\left(X_{r+t}, X_{s+t}\right) = \operatorname{cov}\left(X_{r}, X_{s}\right)$ is not as easy as you might think.

• Unfortunately, you also fail to grasp the concepts. Please see the gospel on time series (IMO): brockwell and davis, time series theory and methods, p81. The series is stationary. What you would realise if you thought about what you just wrote, is that if the 2 was some other number between -1 and 1, then you would have similar difficulties proving the covariance property using your simplistic technique, hence the need to examine the infinite MA representation. Jul 6 '15 at 1:44
• It would appear that if $X_t = 2X_{t-1}+\epsilon_t$, then $\operatorname{var}(X_t) \neq \operatorname{var}(X_{t-1})$ and so the process is neither stationary nor weakly stationary, but then, what do I know? After all, I am not a statistician; just a lowly engineer with a smattering of knowledge of statistics. Rather than question the credibility of your opinion (which I note that you fail to self-deprecatingly qualify as humble), I bow before the superior wisdom of page 81 of the gospel on time series, and wish you all the best in your future studies of the fount of all knowledge. Jul 6 '15 at 12:59

ok, so I know (for example) that $X_t = 2X_{t-1} + e_t$ for iid $e_t$ is stationary.

As far as I know the process that you mentioned is not [weakly]stationary (for stationarity the coefficient must be in $(-1,1)$ interval). Let start with expectation,

Let $E(e_t)=0$ that is very common in time series literature. Also assume that the process starts at a non-zero point. \begin{align} E(X_t)&=2E(X_{t-1})+0\\ &=2(2E(X_{t-2})+0)+0\\ &=\ldots\\ &=2^kE(X_{t-k})\\ &=\vdots \end{align} that is not converging at all. That mean that this process is increasing and then process is not trend stationary.

if you assume that the process starts from zero. then you can find variances with the same procedure as above to see it is not stationary (level stationary).

Edit: For clarification, the above example can be rewritten as:

$X_t = \frac{1}{2} X_{t+1} + u_t$, with $u_t$ some new white noise process. Now we can write down the infinite MA representation of this process, that is,

$X_t = \sum_{j=0}^{\infty} (\frac{1}{2})^{j+1} u_{t+j}$.

Your edit looks convincing. But notice that an standard assumption on time series is that error in future is independent of current time series values. Statistically speaking $Cov(e_t,x_s)=0$ for all $t>s$. Your last equation violates this assumption. On the other hand, that clearly makes the life difficult because there is no record of future to test stationarity and even time series.

• Thanks for your attempt at answering my question. Unfortunately, I don't feel that you grasp the concepts. Jul 3 '15 at 2:12
• Thank you @Jonathan for your reply. I see the your edit and that is fine, excellent. As the policy of SE I should answer the question as it is. let's edit my answer Jul 3 '15 at 7:36

Answer: there is no way to do it, the premise is wrong.

For an AR(1) process $$x_t = \rho x_{t-1} + e_t$$ where $$e_t$$ is a white-noise process, the equation $$\mathbb{V}x_t = \rho^2 \mathbb{V}x_t + \mathbb{V}e_t$$ has no positive finite solution with respect to $$\mathbb{V}x_t$$ if $$\rho^2 \geq 1$$, therefore $$x_t$$ is not weakly stationary and autocovariance $$\mathbb{Cov}(x_t,x_{t+j})$$ does not need to be independent of $$t$$.

Details: Weak stationarity commands:

1. $$\mathbb{E}x_t = \mathbb{E}x_{t+s} \quad \forall s\in \mathbb{Z}$$
2. $$\mathbb{V}x_t <\infty \quad \forall t\in \mathbb{Z}$$
3. $$\mathbb{Cov}(x_t,x_{t+j}) = \mathbb{Cov}(x_{t+s},x_{t+j+s}) \quad \forall s,j\in \mathbb{Z}$$

The latter implies that $$\mathbb{V}x_t = \mathbb{V}x_{t+s} \quad \forall s\in\mathbb{Z}$$.

Take a process $$x_t = \rho x_{t-1} + e_t,$$ where $$e_t \sim WN(0,\sigma^2)$$.

If $$x_t$$ is weakly stationary, the latest condition implies $$\mathbb{V}x_t = \mathbb{V}x_{t-1}$$ and jointly with the process model that $$\mathbb{V}x_t = \rho^2 \mathbb{V}x_t + \mathbb{V}e_t$$

As $$\mathbb{V}e_t > 0$$, if $$\rho^2 \geq 1$$ there is no non-negative solution to the last equation, and the only possibility for $$x_t$$ to have variance at all is for it to be dependent on period of observation, i.e. $$\mathbb{V}x_t \neq \mathbb{V}x_{t-1},$$ which directly contradict the weak stationarity.

Let's consider the covariance between $$X_t$$ and $$X_{t+1}$$ this is $$\text{cov}(X_t,X_{t+1}) = \text{cov}(X_{t},2X_{t}+e_t) = 2 \, \text{var}(X_t)$$

You could generalize this to $$\text{cov}(X_t,X_{t+s}) = 2^s \text{var}(X_t)$$

You assumed that the variance is constant in $$t$$ (because the time-series has a stationary solution), but it is not.

$$\text{var}(X_t) = \text{var}(X_{t-1}+e_t) = \text{var}(X_{t-1}) + \text{var}(e_t) + \underbrace{2\text{cov}(X_{t-1},e_t)}_{\text{= 0 by assumption}} \neq \text{var}(X_{t-1})$$

Inverting the time-direction/causality of the process (like here) is not correctly representing the time-series.

We can indeed rewrite $$X_t = 2X_{t-1} + e_t$$ into $$X_{t-1} = 0.5 X_t - 0.5 e_t$$ or rebrand this $$-0.5 e_t$$ as $$u_{t-1}$$ such that it is $$X_{t-1} = 0.5 X_t + u_{t-1}$$ But this is not similar to an AR1 process with $$\varphi_1 = 0.5$$ because the $$u_{t-1}$$ (which is $$e_t$$ in disguise) correlates with $$X_{t}$$.

This is similar to expressing $$B = A +\epsilon \qquad \text{where: cov(A,\epsilon)=0}$$

in terms of

$$A = B - \epsilon = B + \epsilon^\prime$$

We can not consider $$\text{var}(\epsilon^\prime) = \text{var}(\epsilon)$$ and $$\text{cov}(B,\epsilon^\prime) =0$$.

If we would do this then we could compute the variance like

$$\begin{array}{rcccl} \text{var}(B) &=& \text{var}(A) + \text{var}(\epsilon) + \overbrace{2 \, \text{cov}(A,\epsilon)}^{\text{=0 by assumption}} &=& \text{var}(A) + \text{var}(\epsilon)\\ \text{var}(A) &=& \text{var}(B) + \text{var}(\epsilon^\prime) + 2 \, \text{cov}(B,\epsilon^\prime) &=& \require{cancel} \cancel{ \text{var}(B) + \text{var}(\epsilon^\prime)} \end{array}$$

It would be a contradiction to have both $$\text{var}(B) = \text{var}(A) + \text{var}(\epsilon)$$ and also $$\text{var}(A) = \text{var}(B) + \text{var}(\epsilon)$$