# Why is non-iid noise so important to traditional time-series approaches?

I've been reading the whitepaper that accompanied Facebook's release of Prophet, it's time-series modeling library. One topic the authors drew attention to was that noise was assumed to be iid; they note, that this assumption goes against the grain for traditional time-series solutions, such as ARIMA. Likewise, their solution doesn't account for autocorrelation or moving averages whatsoever.

In general, the Prophet model accounts for piecewise linear (or logistic growth) trends, seasonality, and holiday effects (where seasonality is captured via a fourier series.)

I'm curious, why are autocorrelation, moving average, and non-iid noise emphasized in traditional time-series approaches, such as ARIMA? Wouldn't it be easier to just use a GLM where seasonal controls (whether that be month, week, etc) could be used to augment the overall linear (or logistic trend)?

• Prophet looks for the last turning point then continues the trend to future. The rest is details like seasonality. Is main purpose is forecasting on massive scale, with thousands of series. There’s no way to figure the future turning points in this duration. Yet the trend is the most important aspect of forecasting Oct 12, 2021 at 22:21
• Relevant (especially since very similar threats to inference hold for weak stationarity, such as when an AR(1) root is less than, but close to 1. Oct 26, 2021 at 20:43

This is a very good question. I believe it is very closely related to the question why ARIMA is still one time series analysis and forecasting methods that everyone learns - even though its performance in forecasting is mediocre at best.

My nagging suspicion is that this is not because these methods do a better job at describing reality, and yielding better forecasts. (The proof of the pudding is in the eating, and the proof of the modeling is in the predicting. That, at least, is my opinion.) Rather, it's because time series analysis has historically been the domain of theoretical statisticians and mathematicians. And you can prove theorems about ARIMA and related models. Unit roots! Complex numbers! Characteristic polynomials! And their zeros! Much nicer than methods like exponential smoothing, where the forecasting methods predated a rigorous stochastic model (via state space models) by decades.

Rob Hyndman's "Brief history of forecasting competitions" (2020, IJF) is very enlightening to read in this context. It shows how the earlier forecasting competitions were received by statisticians, who had major difficulties in accepting that simple empirical methods could beat their cherished ARIMA models.

• I disagree with your characterization of arima performance in forecasting. Simple methods were outperforming it in very early competitions, maybe in 1990s. Lately arima based forecasting approaches and other complex methods were performing better than simple methods. That’s what the cited paper stated too if you read it in full Oct 12, 2021 at 22:41
• @Aksakal: complex methods yes, ARIMA no. Looking at Hyndman's paper again (yes, I have read it before), it notes that ARIMA-based methods performed better in the M3 than in earlier competitions. But only in combinations with exponential smoothing and some ForecastPro secret sauce. Since then, the winners of the M4 and M5, and of GEFCOMs, have been quite complex - but anything but ARIMA. Did I miss something? Oct 12, 2021 at 22:48
• Note that "traditional" statistics used to prefer linear methods because the computing power just wasn't there Oct 12, 2021 at 23:15
• "Much nicer than methods like exponential smoothing, where the forecasting methods predated a rigorous stochastic model (via state space models) by decades." - Could you expand upon this? I take it to mean that exponential smoothing, as a forecasting technique, existed decades before state space models were conceived of, legitimizing exponential smoothing... But I've been wrong before! Oct 14, 2021 at 20:43
• Yes, that is exactly what I meant. Exponential smoothing goes back to World War II in updating tracking solutions of British naval gunnery (IIRC), with academic publications by Brown and others starting in the 1950s. The state space formulations only were published 50 years later! Very good references are the reviews by Gardner (1985, JoF and 2006, IJF) and the textbook by Hyndman et al. (2008). Oct 15, 2021 at 6:30

Even in ARIMA models, the noise is still IID; it's just that the model part has an auto-correlation component, a moving-average component, and a differencing component. Now, if you were to take data generated by one of these models and then model it with some simpler model (e.g., lacking the auto-correlation component) then the "noise" from the simpler approximation is going to be non-IID due to the fact that it does not take account of a part of the original model.

• To clarify that I understand, the noise IS iid, just under an ARIMA model, whereas if you used linear regression (for example), the noise would NOT be iid and that's what the basis of the conversation that the Fb Prophet team were referring to? Oct 14, 2021 at 20:39
• Each model assumes that the noise is IID --- as part of diagnostic analysis you can test the residuals to see if this assumption is falsified by the data. In some cases a simpler model is unable to capture relevant aspects of the series and so the residuals display evidence of auto-correlation, etc., which falsifies the IID assumption in the underlying model.
– Ben
Oct 26, 2021 at 20:01

Many statistical methods rely on taking the normalized sample mean of the data and comparing it to a critical value under the null hypothesis. For example you might have $$H_0:\mu=a$$, and your test statistic is $$T=\frac{\bar{X}-a}{\hat{\sigma}/\sqrt{n}}$$ $$\big(\bar{X}$$ = sample mean, $$\hat{\sigma}$$ = square root of the sample variance $$\big)$$, which you then compare to some critical value to reject/fail to reject the null hypothesis.

When there is dependence in the data $$\hat{\sigma}$$ is no longer the correct quantity to use to normalize your test statistic. Instead you would want to use the square root of the long-run variance. Failing to account for this would lead your hypothesis test to have incorrect Type I errors, and more/less power depending on how the long-run variance compares to the variance.

With real data, it is likely not the end of the world if you do not take the dependence into account (and use the square root of the sample variance instead of the square root of the long-run variance). With that said, because the type of hypothesis test I have described is so central to statistics, when possible it is a good idea to incorporate dependence into the errors of time series data.