Stationary processes for AR, MA, ARMA

Question

Depending on the parameters, the AR, MA and ARMA can be either stationary or non-stationary. For instance for an AR(1) process, if $|\phi|<1$, the process is stationary and else it is non-stationary. But why do we restrict ourselves mainly to stationary processes in the theory of AR, MA and ARMA?

I know that ARIMA can be used for non-stationary processes by differentiating the process until it is reasonably stationary but is it possible to fit directly to our non-stationary time series a non-stationary AR, MA or ARMA model?

From my understanding, it seems like if processes were not stationary, we would have a hard time estimating the mean, variance and autocorrelation of the process because they would change at every time step. This is to contrast to the case where the process is stationary, which implies more parsimony in the parameters to estimate for the same amount of data. Hence, if we estimate a non-stationary model, the quality would be very poor compared to the stationary case.

Are there other reasons why we consider only stationary processes for AR, MA and ARMA models?

Answer 1

Short answers:

• We restrict ourself to the stationary region as on the non-stationary one ARMA processes become explosive (that is, they go to infinity)
• It is possible to fit a non-stationary model to time series but that won't be an ARMA model (but it may belong to the family of ARMA models)
• Non-stationary time series need to be at least locally stationary to be modelled. If they are not, we won't have enough observations at each time point to be able to make reasonable estimates. However, if we have a good "sceleton" (see e.g. Tong,H., Non-Linear Time Series) for the series we might be able to extract the non-stationary/nonlinear dynamics from the data and leave a stationary process behind to play with.

Answer 2

Intuition

For AR it depends on what you're going to use the model for, see details below.

It doesn't make sense to estimate the MA part of the ARMA. Remember, if a series follows a unit root every shock persists, forever. Said another way, an error from today or a hundred years ago has the same impact on the series. Since you can't really estimate a MA($\infty$), best to leave out the MA.

Details

Let's focus on a AR(1) model to gain intuition. Let's assume the data is I(1) (i.e. non-stationary). What happens if you estimate a AR(1)? Will the model be any good?

To answer these question you have to know what you want to use the model for. Generally, in time series, you use a model for forecasting or inference.

Forecasting

Yes, we can use the model. We still have consistent coefficient estimates, i.e. the coefficient will be roughly 1. So long as the coefficients are good, our forecasts are good. Word of warning, the process is explosive. This can be seen best by estimating prediction intervals. PI's will not have the usual sideways parabola-shape, rather they will have a sideways absolute value-shape

Inference

Generally no, you can't use the model. Deriving the variance of coefficient estimates is tricky in the presents of a unit root. Intuitively, however, a non-stationary process has no tendency to revert to its mean, implying an infinite variance. Said another way, variance increases to infinity as the number of observation increases, a bad asymptotic result. Further, with a infinite variance, we simply can't reject any null hypotheses.

What about MA?

Let's look at the MA representation of a unit root. We know that a simple AR(1), $y_t = \beta y_{t-1} + u_t$, can be written as a MA($\infty$), $y_t = \sum_{j=0}^\infty \beta^ju_{t-j}$. When $\beta <1$, we're ok, the impact of a shock will eventually die off. Even if $\beta = .9$, $.9^{30}$ is small. However, when $\beta = 1$, there is a big problem! Specifically, shocks never die off. Said another way, a shock today or a hundred years ago would have the same impact on the series, e.g. $1^{30} = 1$ . This makes a MA representation of a unit root process intractable.

Answer 3

I think it would be difficult to even estimate an ARIMA model with non-stationarity, because the tools such as PACF and ACF look different when non-stationarity exists. Knowing what was the right order, the AR and MA levels, would be difficult at least given the classical presentations used to identify these.