3
$\begingroup$

Rob Hyndman states:

"The paper describing the competition [M] (Makridakis et al, 1982) had a profound effect on forecasting research. It caused researchers to: ... treat forecasting as a different problem from time series analysis"

and

"The paper describing the competition (Makridakis et al. 1982) had a profound effect on forecasting research. It caused researchers to: focus attention on what models produced good forecasts, rather than on the mathematical properties of those models; ...; treat forecasting as a different problem from time series analysis. These now seem like common-sense to forecasters, but they were revolutionary ideas in 1982. Even today, I often have to explain to other academics why forecasting is not just an application of time series analysis.".

The author indirectly states that time series analysis is focused on the mathematical properties of models rather than on models that produce the best forecasts. Restricting our focus to univariate time series analysis with ARIMA, what does it mean for ARIMA time series analysis to be focused on the mathematical properties of models (as opposed to focused on best forecasts)?

Hyndman, R. J. (2020). A brief history of forecasting competitions. International Journal of Forecasting, 36(1), 7–14.

Improve this question
$\endgroup$
2
  • $\begingroup$ re "indirectly states:"I think you have read more into this quotation than it said or meant. It is referring to certain forms of time series analysis popularized in the 60's and 70's: namely, ARIMA models and their relatives. Understanding that, how does this question differ (if at all) from your previous one about this topic? $\endgroup$
    – whuber ♦
    Commented Jan 5, 2022 at 21:16
  • $\begingroup$ Maybe I read too much into it ... Note this question is specifically about ARIMA and about the mathematical properties of ARIMA. Put it differently, how is ARIMA time series not about producing the model that generates best forecasts? At which steps in the analysis are we choosing to focus on models with good mathematical properties as opposed to on models that produce the best forecasts? I may end up reaching out to prof. Hyndman for comment, after all. $\endgroup$
    – ColorStatistics
    Commented Jan 5, 2022 at 21:33

1 Answer 1

Reset to default
3
$\begingroup$

I don't think there is (or better: was, at the time of the first forecasting competitions) a true opposition between getting good forecasts and focusing on mathematical properties.

Rather, my reading of the literature would be as follows: statisticians and mathematicians looked at time series and very understandably focused on the data generating process. It's probably a fair assessment to say that this is a natural first reaction for statisticians. They quickly thought about AR and MA processes, combined these, added integration and ended up with ARIMA processes.

These were, on the one hand, fertile grounds for mathematical analysis. You could prove all kinds of theorems about stationarity, unit roots, estimation, identification of correct ARIMA model orders and so on. And on the other hand, if your original DGP was ARIMA, then the correct ARIMA model would give you MSE-optimal forecasts. This in turn increased interest in identifying that true ARIMA model.

More precisely, as Richard Hardy points out this would hold under perfect estimation precision, i.e. never in reality. Under more realistic imperfect estimation precision, an ARIMA model with lag orders different from these of the true DGP may generate more precise forecasts due to the bias-variance trade-off. Yes, statisticians were aware of this - but this again offers interesting avenues for research, in terms of asymptotics, and of quantifying the discrepancy between the MSE of true and misspecified models.

Extensions to (G)ARCH, VAR, VECM etc. followed, and there is again indeed a lot of ground to cover here, witness any time series analysis textbook.

There was no point where people decided to focus on mathematical elegance over good forecasts. Rather, it was a point of having found a framework in which both mathematical elegance led to good forecasts.

The problem was (again, let me emphasize that this is only my impression) that people forgot about that crucial assumption that the original time series was generated by an ARIMA process, or that it could at least be well enough approximated by one. If you want to be uncharitable, it's a case of having a hammer and all (forecasting) problems looking like nails, or of searching for your lost keys under an ARIMA lamp post, because the light is best there. This, it seems to me, was the intellectual background against which we have to see the early forecasting competitions in the 1980s, which for the first time focused on forecasting performance in a model- and DGP-agnostic way. The hammer-wielders saw people with screwdrivers and did not understand how they were going to get their nails into the wall... until they saw that there were screws among the nails they had been picking out of the toolbox.

Let me emphasize once more that this is only my understanding, not based on any particular research.

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ if your original DGP was ARIMA, then the correct ARIMA model would of course give you MSE-optimal forecasts, which in turn increased interest in identifying that true ARIMA model. To be a little more precise, this is strictly true only under perfect estimation precision, i.e. never in reality. Under imperfect estimation precision that we encounter in all applications, an ARMA model with lag orders $p$, $q$ different from these of the DGP may generate more precise forecasts due to the bias-variance trade-off. Otherwise a nice answer (I also liked hammers and screws and the rest)! $\endgroup$
    – Richard Hardy
    Commented Jan 6, 2022 at 9:09
  • $\begingroup$ @RichardHardy: thank you for reading my posts precisely enough to identify and point out my imprecisions ;-) $\endgroup$
    – Stephan Kolassa
    Commented Jan 6, 2022 at 9:15
  • $\begingroup$ I like to learn things, and you are a source to follow. I think it would be a good idea to make a small edit to your post to reflect my point, as not everyone reads comments. Thank you. $\endgroup$
    – Richard Hardy
    Commented Jan 6, 2022 at 9:53
  • $\begingroup$ @RichardHardy: good point again, I did so, and credited your comment. Thanks once more! $\endgroup$
    – Stephan Kolassa
    Commented Jan 6, 2022 at 10:09
  • 1
    $\begingroup$ That is a good summary. Note that combining forecasts is usually beneficial, but there are still a lot of forecasters who use single models, essentially because they are so much easier to understand and to communicate. But again, these may have little mathematical underpinning (as I wrote elsewhere, the mathematical foundations for exponential smoothing only happened 50 years after smoothing was introduced, and performed so well as to be surprising for statisticians). $\endgroup$
    – Stephan Kolassa
    Commented Jan 6, 2022 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.