Linear Regression & SARIMAX in Time Series Forecasting

Question

I am just beginning to learn about ARIMA and have been interested in using linear regression with ARIMA errors (I have been reading through these sections in FPP), and have some questions:

1. Are you building the ARIMA (or SARIMA) model on the residuals of the linear regression model during training, or on the values of your target variable during training? I have seen it done both ways and assumed it would be built on the residuals.

2. A slightly more a general question, but why would you not use SARIMAX over linear regression when working with time series? Can there be any advantage of not using it (i.e. is it possible the model would perform worse, assuming you picked the optimal orders for the ARIMA model)? If the residuals are normally distributed, had no autocorrelation, stationary etc. from your linear regression model, then would there be no point in using (S)ARIMA? (I understand if the residuals were genuinely white noise, then there would be no point).

Appreciate any insight into this!

Answer 1

1. There is a lot of confusion about ARIMAX vs. regression with ARIMA errors. Rob Hyndman's R packages use the latter. Take a look at his The ARIMAX model muddle.

2. I can think of a number of cases where modeling regression residuals in a time series situation (i.e., regression with ARIMA errors, per above) would be problematic:

   • Missing data. ARIMA is not happy about missing data, so you would need to impute. If a lot of data is missing (e.g., retail data with long periods where a product is simply not sold), this becomes very dubious indeed.
   • Long seasonal periods. SARIMA has problems in this case, it may either take a long time, or fail to converge altogether. For example, retail data often exhibits yearly seasonality, so daily data has a seasonal period of 365.
   • Short time series. If you have observed a small number of seasonal cycles, SARIMA becomes rather dubious to use. Retail data, for instance, often has short time series because of the frequent introduction of new products.
   • multiple-seasonalities, which SARIMA can't model at all (unless you again add Fourier terms, so you are back to a regression). You could of course use specialized methods for this. Retail data, to use a random example, often exhibits both year-over-year and week-over-week seasonalities.

   I will let you guess what kind of time series I forecast, and whether I use regression with ARIMA errors.