I am just beginning to learn about ARIMA and have been interested in using linear regression with ARIMA errors (I have been reading through these sections in FPP), and have some questions:

  1. Are you building the ARIMA (or SARIMA) model on the residuals of the linear regression model during training, or on the values of your target variable during training? I have seen it done both ways and assumed it would be built on the residuals.

  2. A slightly more a general question, but why would you not use SARIMAX over linear regression when working with time series? Can there be any advantage of not using it (i.e. is it possible the model would perform worse, assuming you picked the optimal orders for the ARIMA model)? If the residuals are normally distributed, had no autocorrelation, stationary etc. from your linear regression model, then would there be no point in using (S)ARIMA? (I understand if the residuals were genuinely white noise, then there would be no point).

Appreciate any insight into this!


1 Answer 1

  1. There is a lot of confusion about ARIMAX vs. regression with ARIMA errors. Rob Hyndman's R packages use the latter. Take a look at his The ARIMAX model muddle.

  2. I can think of a number of cases where modeling regression residuals in a time series situation (i.e., regression with ARIMA errors, per above) would be problematic:

    • Missing data. ARIMA is not happy about missing data, so you would need to impute. If a lot of data is missing (e.g., retail data with long periods where a product is simply not sold), this becomes very dubious indeed.
    • Long seasonal periods. SARIMA has problems in this case, it may either take a long time, or fail to converge altogether. For example, retail data often exhibits yearly seasonality, so daily data has a seasonal period of 365.
    • Short time series. If you have observed a small number of seasonal cycles, SARIMA becomes rather dubious to use. Retail data, for instance, often has short time series because of the frequent introduction of new products.
    • , which SARIMA can't model at all (unless you again add Fourier terms, so you are back to a regression). You could of course use specialized methods for this. Retail data, to use a random example, often exhibits both year-over-year and week-over-week seasonalities.

    I will let you guess what kind of time series I forecast, and whether I use regression with ARIMA errors.

  • $\begingroup$ @Stephen Thanks for the response! That makes sense, a quick follow up point on 2) couldn't you use ARIMA without a seasonal component, if the errors didn't exhibit a simple seasonality, i.e. wouldn't information from yesterday, the day before etc.. still be of use? Thanks once again! $\endgroup$
    – b247
    Jun 25, 2022 at 14:15
  • 1
    $\begingroup$ Yes, if your series is not seasonal, then three of my bullet points are not relevant. (Missing data are still a problem.) $\endgroup$ Jun 25, 2022 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.