# ARMA models and predictive learning

It is well known that, in Time Series, ARMA models was proposed for make forecast about some $$y_t$$ based on past values of the process itself.

However this is not the only possible way for make forecast about some target variable $$y_t$$. Indeed is possible that past values of other process like $$x_t$$ can help. Indeed the concept of Granger causality (Granger predictability would be a better name) consider precisely this possibility.

Now limiting our interest into stationary time series (wide sense stationary) we can consider all above as special case of standard linear regression analysis. Indeed we can reduce ARMAs to a pure autoregressive representations too.

Moreover, also in predictive learning literature is common to use linear regression for make forecast. In this literature is common to consider many predictors, not only past values of the target variable. In order to select the best subset of predictors the selection criteria matters (for instance: BIC, AIC, sequential hypothesis testing, LASSO). Strategies like these follow the Granger predictability idea.

Now, in: A Guide to Modern Econometrics – Verbeek; Wiley (2017). ARMA models are presented. However at Par 8.12: What about Multivariate Models? is argued that in economics most series are related. For example we can have

$$y_t = \beta x_t + \epsilon_t$$

but if $$x_t$$ is representable as an ARMA process, then $$y_t$$ has an ARMA representation too (sum of ARMAs is ARMA). Author said: “Thus, the fact that two variables are related does not imply that a pure times series approach is invalid.”.

Now, it seems me that this story can be extended for many variables where for each of them something like follows hold

$$y_t = X_t ' \beta + \epsilon_t$$

where $$X_t$$ is a vector of variables at time “t”, but any $$y_t$$ maintain his ARMA representation. Better, not only $$y_t$$ maintain his ARMA representation but it come from relations with $$X_t$$ or, at least, match with them.

However if this story hold, from ARMA theory, we can stay focused only on ACF and PACF of $$y_t$$; we can forget any other potential predictors. We can skip the $$X_t$$ part. Granger predictability cannot work. Common practice of predictive learning become a waste of time!

So, them seems me to strong conclusions. However, at logic level, I don't find mistakes. What go wrong above?

Note: interesting to note that in machine learning literature ARMA models are not considered so much; for example in the very common manual: The Elements of Statistical Learning, Data Mining, Inference, and Prediction - Hastie Tibshirani Friedman; Springer (2008); ARMA models are completely not mentioned. Conversely in Time Series literature the common tools in predictive learning, as stepwise selection with some criteria, are not considered and most pages are spent in ARMA process and related concepts. For example we can see in: Introduction to Time Series and Forecasting - Brockwell and Davis; Springer (2016).

• Hi: I'm not sure what you're asking but think of a VAR. That has selection criteria ( how many lags ? ) and it's a time series model and it's used for prediction and one can use granger causaility in the VAR framework. – mlofton Nov 21 '20 at 14:54
• So the question is "since most economic time series are related can we use simple ARMA instead of VARMA models?"? If so, then it depends: VAR models consider multiple variables and can be reformulated as VECM if there is cointegration, in order to model (economic) equilibria between variables and short-term deviations. On the other hand, VAR has large estimation error due to large number of parameters, which very often offsets its advantages. Regarding your note: machine learning excels with non-linear relations while "standard" econometric models (lasso, ARMA etc) best work if they're linear. – PaulG Nov 21 '20 at 15:23
• Even if several predictors are potentially involved, I spoke precisely of univariate models only. No systems, no VAR. We can read all above as something like a race between: pure ARMA against linear regression with all (lagged) predictors we want. Based on ref I cited seems me that the second strategy boils down to the first (ARMA). It is so? – markowitz Nov 21 '20 at 15:28
• @PaulG, I excluded integrated variables for simplicity. – markowitz Nov 21 '20 at 16:20
• I modified a bit the explanation for clarify my point. – markowitz Nov 22 '20 at 11:53