Timeline for Selecting ARIMA Order using Rolling Forecast
Current License: CC BY-SA 3.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 13, 2017 at 18:36 | comment | added | Richard Hardy | Agree, sound logical. | |
| Feb 13, 2017 at 18:30 | comment | added | Aksakal | If you deal with the business data, e.g. retail revenues, then it's easy to see how time separated holdout samples would be an issue because there could be changes in the business between sub samples. | |
| Feb 13, 2017 at 18:19 | comment | added | Richard Hardy | OK, in the sense that leave-one-out cross validation and AIC are asymptotically equivalent in terms of model selection, unless (if I am not mistaken) we misspecify the likelihood which causes AIC to misbehave. But I maintain that there is a clear difference in model evaluation when doing out-of-sample vs. in-sample forecast comparison. In-sample comparison can be way too optimistic while out-of-sample (using rolling windows) will be much more realistic. | |
| Feb 13, 2017 at 18:15 | comment | added | Aksakal | @RichardHardy there is a difference, but we blur it when using cross validation the way you describe to a point where it become almost indistinguishable in terms of power for model selection. the ustility of out of sample is in dealing with overfitting, and in your procedure it doesn't help much in this regard | |
| Feb 13, 2017 at 17:52 | comment | added | Richard Hardy | Also, what do you mean by your last paragraph? Clearly, there is a difference between in-sample and out-of-sample model validation (just like in cross-sectional setting). | |
| Feb 13, 2017 at 17:51 | comment | added | Richard Hardy | ACF/PACF work fine in simple cases like AR(p) or MA(q), but not for ARMA(p,q). Even so the typical model identification based on ACF/PACF would yield a too complex model, e.g. more complex than AIC-based selection which should be preferred when forecasting due to the efficiency property of the AIC. Rolling windows could be superior when the error distribution is quite different from the assumed distribution in which case AIC can be distorted, and also when the data generating process evolves over time (which will not be captured by fitting a model on the whole sample). | |
| Feb 13, 2017 at 15:58 | history | answered | Aksakal | CC BY-SA 3.0 |