# The explosive AR(1) process with $\varphi>1$, where was this first represented as a stationary, but non-causal, time-series?

According to this question and answer Explosive AR(MA) processes are stationary? the AR(1) process (with $$e_t$$ white noise):

$$X_{t}=\varphi X_{t-1}+e_{t} \qquad , e_t \sim WN(0,\sigma)$$

is a stationary process if $$\varphi>1$$ because it can be rewritten as

$$X_t=\sum_{k=0}^\infty {\varphi}^{-k}u_{t+k}$$

But now the variable $$X_t$$ depends on the future.

I wonder where this representation (which I remember having seen in several places) and the derivation originally comes from.

I am confused about the derivation, and I wonder how it works. When I try to do the derivation myself I am failing.

I can rewrite the process $$X_{t+1}=\varphi X_{t}+e_{t+1}$$ as $$X_{t}= \varphi^{-1} X_{t+1} -\varphi^{-1} e_{t+1}$$ and replacing $$\varphi^{-1} e_{t+1}$$ by $$u_{t}$$ it becomes $$X_{t}= \varphi^{-1} X_{t+1} + u_{t}$$ such that the expression is 'like' another AR(1) process but in reverse time and now the coefficient is below 1 so it seemingly is stationary (*).

From the above it would follow indeed that $$X_t=\sum_{k=0}^\infty {\varphi}^{-k}u_{t+k}$$ (*) But the $$u_t$$ is not independent from $$X_{t+1}$$, because it is actually $$e_{t+1}$$ times a negative constant.

• "...because it can be rewritten as..." is not really correct and suggests possible misconception. Rather, given the (stochastic) sequence $(\epsilon_t)$, the stochastic difference equation...[AR(1) with root inside unit circle] has a non-causal stationary solution.... "But the $u_t$ is not independent from $X_{t+1}$"---actually it is, if you look at the solution itself:$X_{t+1}$ is a function of $\epsilon_{t+2}, \epsilon_{t+3}, \cdots$, therefore independent of $\epsilon_{t+1}$ (assuming $(\epsilon_t)$ is i.i.d.). Commented Oct 29, 2020 at 1:58
• @Michael $X_{t+1}$ is by definition a function of $e_{t+1}+e_{t}+e_{t-1}+e_{t-2}...$, and therefore dependent on $e_{t+1}$. So this reversed equation is not similar to the typical AR(1) where the increments are white noise that is independent from the current value. Commented Oct 29, 2020 at 8:00
• It is maybe like reversing the first shot in a billiards game. In the first shot the balls that are initialy placed in a triangle and get randomly dispersed, the order decreases. When we invert time then the order is not similarly increasing, because the random terms are 'predetermined'. This time-reversal only makes sense when you reset the white noise, which is not truly going backwards in time (at least not in the sense of playing a movie backwards). Commented Oct 29, 2020 at 8:16
• I get that if you reverse time you get a different, stationary, process, but I do not see how this solution is stationary (as a movie going backwards in which case billiard balls get perfectly back into place, which is different as turning the physical laws and causation backwards in which chaos will increase and you get a different movie, and different error terms, when turning the laws backwards). So this makes me wonder what the history of this approach is, where and why did they do this the first time. Commented Oct 29, 2020 at 17:21
• As for the historical question---I believe Mann and Wald (1943) already considered non-causal AR(1) case, among other examples. Perhaps one can find references even earlier. Commented Oct 29, 2020 at 17:38

The question suggests some basic confusion between the equation and the solution

## The Equation

Let $${\varphi} > 1$$. Consider the following (infinite) system of equations---one equation for each $$t\in \mathbb{Z}$$: $$X_{t}=\varphi X_{t-1}+e_{t}, \mbox{ where } e_t \sim WN(0,\sigma), \;\; t \in \mathbb{Z}. \quad (*)$$

Definition Given $$e_t \sim WN(0,\sigma)$$, a sequence of random variables $$\{ X_t \}_{t\in \mathbb{Z}}$$ is said to be a solution of $$(*)$$ if, for each $$t$$, $$X_{t}=\varphi X_{t-1}+e_{t},$$ with probability 1.

## The Solution

Define $$X_t= - \sum_{k=1}^\infty {\varphi}^{-k}e_{t+k},$$ for each $$t$$.

1. $$X_t$$ is well-defined: The sequence of partial sums $$X_{t,m} = - \sum_{k=1}^m {\varphi}^{-k}e_{t+k}, \;\; m \geq 1$$ is a Cauchy sequence in the Hilbert space $$L^2$$, and therefore converges in $$L^2$$. $$L^2$$ convergence implies convergence in probability (although not necessarily almost surely). By definition, for each $$t$$, $$X_t$$ is the $$L^2$$/probability-limit of $$(X_{t,m})$$ as $$m \rightarrow \infty$$.

2. $$\{ X_t \}$$ is, trivially, weakly stationary. (Any MA$$(\infty)$$ series with absolutely summable coefficients is weakly stationary.)

3. $$\{ X_t \}_{t\in \mathbb{Z}}$$ is a solution of $$(*)$$, as can be verified directly by substitution into $$(*)$$.

This is a special case of how one would obtain a solution to an ARMA model: first guess/derive an MA$$(\infty)$$ expression, show that it is well-defined, then verify it's an actual solution.

$$\;$$

...But the $$\epsilon_t$$ is not independent from $$X_{t}$$...

This impression perhaps results from confusing the equation and the solution. Consider the actual solution: $$\varphi X_{t-1} + e_t = \varphi \cdot \left( - \sum_{k=1}^\infty {\varphi}^{-k}e_{t+k-1} \right) + e_t,$$ the right-hand side is exactly $$- \sum_{k=1}^\infty {\varphi}^{-k}e_{t+k}$$, which is $$X_t$$ (we just verified Point #3 above). Notice how $$e_t$$ cancels and actually doesn't show up in $$X_t$$.

$$\;$$

...where this...derivation originally comes from...

I believe Mann and Wald (1943) already considered non-causal AR(1) case, among other examples. Perhaps one can find references even earlier. Certainly by the time of Box and Jenkins this is well-known.

## Further Comment

The non-causal solution is typically excluded from the stationary AR(1) model because:

1. It is un-physical.

2. Assume that $$(e_t)$$ is, say, Gaussian white noise. Then, for every non-causal solution, there exists a causal solution that is observationally equivalent, i.e. the two solutions would be equal as probability measures on $$\mathbb{R}^{\mathbb{Z}}$$. In other words, a stationary AR(1) model that includes both causal and non-causal cases is un-indentified. Even if the non-causal solution is physical, one cannot distinguish it from a causal counterpart from data. For example, if innovation variance $$\sigma^2 =1$$, then the causal counterpart is causal solution to AR(1) equation with coefficient $$\frac{1}{\varphi}$$ and $$\sigma^2 =\frac{1}{\varphi^2}$$.

• A detailed answer following a fascinating discussion in the comments. I love it! Commented Oct 29, 2020 at 20:21
• I like this answer for it's clear explanation and especially the reference to Mann and Wald. Commented Oct 29, 2020 at 20:25
• You are saying that the non-causal solution is excluded because it is unphysical. I believe that this is how I look at the equation and what is maybe the source of confusion that leads to my impression (if it is actually confusion). I can see that your solution works by plugging it into the equation, but I regard this as an unreal solution because I see the equation as some sort of physical process for which the causality can not be reversed in time... Commented Oct 29, 2020 at 20:33
• ... If I compare the variance of $X_t$ as function of the variance of $X_{t+1}$, then I believe the solution can not be used to determine the variance. In my eyes the relationship of variances $$Var(X_t) = \varphi^{-1}Var(X_{t+1}) + \varphi^{-1} Var(\epsilon_t)$$ which would follow from that solution, is not right. Or contradicts with $$Var(X_{t+1}) = \varphi Var(X_{t}) + Var(\epsilon_{t+1})$$ It is a difference whether we see $X_t$ as a function of $X_{t+1} + \text{noise}$ or $X_{t+1}$ as a function of $X_t + \text{noise}$. But this difference in direction is not clear in the equation. Commented Oct 29, 2020 at 20:43
• A simpler analogy. The equation $$A-B = \epsilon$$ with $\epsilon$ standard normal distributed and $A,B$ a bivariate normal distribution with covariance some value $\phi$ has indeed two solutions (due to the symmetry). But in the context of a causal model only one of them is 'accepted' or physical. E.g. when we speak of 'A is equal to B with noise added to it' then only one of the two solutions makes sense. Commented Oct 29, 2020 at 20:53

Re-arranging your first equation and increasing the index by one gives the "reverse" AR(1) form:

$$X_{t} = \frac{1}{\varphi} X_{t+1} - \frac{e_{t+1}}{\varphi}.$$

Suppose you now define the observable values using the filter:

$$X_t = - \sum_{k=1}^\infty \frac{e_{t+k}}{\varphi^k}.$$

You can confirm by substitution that both the original AR(1) form and the reversed form hold in this case. As pointed out in the excellent answer by Michael, this means that the model is not identified unless we exclude this solution by definition.

• I like your brief answer. I can see that this 'other' solution occurs because the equations do not necessarily have a causal interpretation (and my confusion with it is that I did give the equations a causal interpretation, as a formula for an iterative scheme). It is sort of like computing the size of an object with a particular area as the solution of $l^2\propto A$, which allows negative size. Commented Oct 29, 2020 at 23:04
• But I still wonder about my main question where this alternative solution originally comes from. Commented Oct 29, 2020 at 23:06
• Yes, fair enough. I'm sorry I was unable to answer that part --- my knowledge of the original time-series books is thin.
– Ben
Commented Oct 29, 2020 at 23:43

It is interesting to notice that the explosive AR(1) process $$X_t-\phi X_{t-1}=Z_t$$ with $$|\phi|>1$$, $$\phi\in\mathbb{R}$$, can be expressed as a causal AR(1) process of the form \begin{align} X_t-\phi^{-1} X_{t-1}=\widetilde{Z}_t \end{align} for a new white noise $$(\widetilde{Z}_t:t\in\mathbb{Z}\}$$.

Indeed, as noted by several of the answers (and the OP itself), for each $$t\in\mathbb{Z}$$ \begin{align} X_t=-\sum^\infty_{n=1}\phi^{-n}Z_{t+n}\tag{0}\label{zero} \end{align} whence we obtain \begin{align} X_t-\phi^{-1}X_{t-1}=\phi^{-2}Z_t +\sum^\infty_{n=1}(\phi^{-2}-1)\phi^{-n}Z_{t+n}=: \widetilde{Z}_t\tag{1}\label{one} \end{align} It remains to show that $$(\widetilde{Z}_t:t\in\mathbb{Z})$$ is $$WN(0,\tilde{\sigma}^2)$$ for some $$\tilde{\sigma}^2>0$$. Let $$\gamma_Z(h)=\sigma^2\delta_{0,h}$$ be the correlation function of the white noise $$Z=WN(0,\sigma^2)$$. Writing $$\widetilde{Z}_t=\sum_j\beta_jZ_{t+j}$$, we have that $$\mathbb{E}[\widetilde{Z}_t]=0$$ for all $$t$$ and, \begin{align}\mathbb{E}[\widetilde{Z}_{t+h}\widetilde{Z}_t]&=\sum_{j, k}\beta_j\beta_k\mathbb{E}[Z_{t+h+j}Z_{t+k}]=\sum_{j, k}\beta_j\beta_k\gamma_Z(h+j-k)\\ &=\sum^\infty_{j=0}\phi_j\phi_{j+h}\sigma^2\tag{2}\label{two} \end{align} When $$h=0$$, \begin{align} \tilde{\sigma}^2=\gamma_{\widetilde{Z}}(0)&=\sigma^2\Big(\phi^{-4}+(\phi^{-2}-1)^2\sum^\infty_{n=1}\phi^{-2n}\Big)\\ &=\sigma^2\big( \phi^{-4}-(\phi^{-2}-1)\phi^{-2}\big)=\sigma^{2}\phi^{-2}>0 \end{align} For $$h\neq0$$ \begin{align}\sigma^{-2}\gamma_{\widetilde{Z}}(h)&=\phi^{-2}(\phi^{-2}-1)\phi^{-h} +\sum^\infty_{n=1}(\phi^{-2}-1)(\phi^{-2}-1)\phi^{-n}\phi^{-(n+h)}\\ &=\phi^{-(2+h)}(\phi^{-2}-1)+(\phi^{-2}-1)^2\phi^{-h}\frac{\phi^{-2}}{1-\phi^{-2}}=0 \end{align}

The validity and convergence of the series \eqref{zero}, \eqref{one} and \eqref{two} is consequence of the following exercise on dominated convergence:

Proposition A: Suppose $$(\psi_j:j\in\mathbb{Z})\subset\mathbb{C}$$ satisfies $$\sum_j|\phi_j|<\infty$$, and $$(X_j,\,Y_j:j\in\mathbb{Z})\subset L_2(\mathbb{P})$$ with $$\mathbb{E}[Y_j]=\mathbb{E}[X_k]=\mu$$ for all $$j,k$$, and $$\sup_j\{\|X_j\|_2,\|Y_j\|_2\}<\infty$$. Then \begin{align} U=\sum_j\psi_jX_j,\qquad V=\sum_j\psi_jY_j \end{align} converge a.s. and in $$L_2(\mathbb{P})$$; furthermore, the double series $$m=\mathbb{E}[U]=\sum_{j,k}\phi_j\mu=\mathbb{E}[V]$$ converges absolutely, and $$\mathbb{E}[(U-m)\overline{(V-m)}]=\sum_{j,k}\phi_j\overline{\phi_k}\mathbb{E}[(X_j-\mu)\overline{(Y_k-\mu)}]$$

• I don't follow your first sentence with the two equations. Commented Mar 19 at 7:53
• Do you mean to re-express it as a seemingly non-explosive (causal) AR process, but the $\tilde{Z}_t$, albeit being WN, may not need to be independent from the time series $X_t$. Does white noise guarantee that $X_{t-1}$ is uncorrelated with $u_t$? Commented Mar 19 at 8:03
• @SextusEmpiricus: Following Rockwell and Davis, Time Series: Theory and Methods, I meant to say that $X_t$ can be expressed as an AR(1) process that is causal function of a (different) white noise $(\tilde{Z}_t:t\in\mathbb{Z})$. This is problem 3.3 of the aforementioned book (page 110). Commented Mar 19 at 15:37
• I find it confusing. So if it is a causal proces then we can write it as a definition $X_t := \phi^{-1} X_{t-1} + \tilde{Z}_t$ and here $\tilde{Z}_t$ may be white noise, but it is correlated with future values of $X_s, s>t$. Commented Mar 19 at 17:46
• Mind blowing... $\phi>1$ suggests explosive behavior while $|\phi|<1$ suggests stationary behavior. If we can express a process in both ways, what kind of behavior can we expect? I do not think we can have both types at once. Commented Mar 19 at 21:02

... the AR(1) process (with $$e_t$$ white noise):

$$X_{t}=\varphi X_{t-1}+e_{t} \qquad , e_t \sim WN(0,\sigma)$$

is a stationary process if $$\varphi>1$$ because ...

It seems me not possible as showed there: https://en.wikipedia.org/wiki/Autoregressive_model#Example:_An_AR(1)_process

for wide sense stationarity $$-1 < \varphi < 1$$ must hold.

Moreover, maybe I lose something here but it seems me that not only the process above cannot be stationary but it is entirely impossible and/or bad defined. This because if we have an autoregressive process, we do not stay in a situation like $$Y=\theta Z+u$$ where $$Z$$ and $$u$$ can be two unrestricted random variables and $$\theta$$ an unrestricted parameter.

In a regression residuals and parameters are not free terms, given dependent and independent/s variables, they are given too.

So, in AR(1) case it is possible to show that $$-1 \leq \varphi \leq 1$$ must hold; like autocorrelation.

Moreover if we assume that $$e_t$$ (residuals) are white noise process ... we make a restriction on $$X_t$$ process too. If in the data we estimate an AR(1) and $$e_t$$ result as autocorrelated ... the assumption/restriction do not hold ... AR(1) is not a good specification.