Do I need to take the residuals of the ARMA fit for my linear regression?

Question

I am not used to time series econometrics. If I understand well, before conducting a linear regression (the most basic possible), we need to ensure the stationarity of our data - particularly with (Augmented) Dickey-Fuller tests. Then, we need to fit the optimal ARMA(p,q) model to our stationary data.

Once we've fitted ARMA model to the data, we can do forecasts, however I don't really understand how we can use them in linear regressions. Do we need to take the residuals of the ARMA process (our series without AR or MA influence) as our "new" variables for our regression ? Do we loose much information by doing so ?

I understand that we fit the ARMA process on regression residuals to check their stationarity and ensure that we did not do a spurious regression. But I am not sure on how to use the ARMA before the regression !

Answer 1

ARMA is a pure time series model, i.e. it doesn't have exogenous variables $X_t$ unlike a typical regression $y_t=X_t\beta+\varepsilon_t$. In a generic formulation of ARMA such as $\phi(B)y_t=\theta(B)\varepsilon_t$ the exogenous variables $X_t$ are nowhere to be seen.

There are a few ways to marry these two models. One is called ARIMAX: $$\phi(B)y_t=X_t\beta+\theta(B)\varepsilon_t$$ and the other is regARIMA: $$y_t=X_t\beta+u_t\\\phi(B)u_t=\theta(B)\varepsilon_t$$

The latter formulation can be easier to understand for you as it states that we have a linear regression where the errors are ARIMA, which is similar to how you described your understanding. Both approaches are estimated in MLE framework where exogenous coefficients $\beta$ and ARMA coefficients are estimated simultaneously, not in the two step approach that you suggest.

If you didn't have moving average part in ARMA, then the ARIMAX can be estimated in a linear regression with OLS: $$\phi(B)y_t=X_t\beta+\varepsilon_t$$ $$y_t=\phi_1 y_{t-1}+\phi_2 y_{t-2}+\dots+X_t\beta+\varepsilon_t$$