4
$\begingroup$

Following up on a discussion in a previous thread:

Are there any situations where adding too much data to a forecasting model is counterproductive, in the sense that it reduces the forecasting accuracy of the model?

Specifically, if I have a univariate time series model, is there such thing as using too much history? Are there situations where I have 20 years of monthly data, but I would be better off using only the most recent 5 years of data, since older data would reduce the accuracy of the model.

For example if there was a structural break in the time series, does feeding data from before the break to the time series somehow throw the model off?

Or does the ML truism that "more data = better accuracy" hold for time series as well?

$\endgroup$
  • $\begingroup$ As always, it depends. Often some pieces of data are of lower quality than others: throwing lots of low-quality data at a problem does not necessarily improve the solution. $\endgroup$ – whuber Apr 24 '18 at 16:27
  • $\begingroup$ Do you mean adding new variables or increasing the sample size of the existing variables? $\endgroup$ – Richard Hardy Apr 24 '18 at 17:42
  • $\begingroup$ @RichardHardy increasing the sample size. See edit. $\endgroup$ – Reinstate Monica Apr 24 '18 at 17:48
  • $\begingroup$ @RichardHardy stats.stackexchange.com/questions/343848/… $\endgroup$ – Reinstate Monica May 1 '18 at 17:46
4
$\begingroup$

If the time series data are coming from an unchanging data generating process (DGP), having a longer time series (larger sample size) can only be beneficial. Most models will be fit with greater accuracy on more data and thus will generate better forecasts. (A counterexample would be a spurious regression model for a pair of random walks. There, having a larger sample will not necessarily lead to a better estimate of the slope that is actually zero. Hence, if the model is terrible to begin with, a longer sample might not help in improving it.)

If the time series are coming from a DGP that is evolving over time and the model does not account for that, more data may either be beneficial or detrimental. A longer history may be a more irrelevant history to the extent that it would harm forecasting performance. This depends on how quickly the DGP is changing and how much history you already have, and how much more history you are adding.

[D]oes feeding data from before the break to the time series somehow throw the model off?

Yes, it very well may, unless the break is minuscule and the historical data is relevant enough for the new period.

$\endgroup$
  • $\begingroup$ +1 A formal approach that considers whether adding observations prior to a structural change improves the forecasts or not is: Pesaran MH, Timmermann A (2002). "Market Timing and Return Prediction under Model Instability." Journal of Empirical Finance, 9, 495–510. doi:10.1016/s0927-5398(02)00007-5 $\endgroup$ – Achim Zeileis Apr 28 '18 at 10:49
1
$\begingroup$

Thanks for the question as it leads to a teaching moment ....

An often overlooked caveat when dealing with data is the assumption that the parameters to be optimized are invariant . In practice with time series data , the question should be asked "have the parameters changed over time" or equivalently do we have too much data ?. Tong in 1980 considered this issue enter image description here with the following idea in mind enter image description here.

Following is a data set from his work suggesting how to deal with this issue.

enter image description here with acf here enter image description here

If one identified a model based upon the 40 values , we get enter image description here and a residual plot of enter image description here . The residual plot suggests model deficiency which might be treated by incorporating variance change detection ala Tsay http://docplayer.net/12080848-Outliers-level-shifts-and-variance-changes-in-time-series.html or more simply parameter change detection via a sequential use of the Chow test for constant parameters https://en.wikipedia.org/wiki/Chow_test where the grouping is based upon a search .

The acf of the residuals does not warn us about the violation .enter image description here

If we estimate the AR(1) model separately for different possible grouping , we get enter image description here suggesting the optimal partitioning is 1-22 vs 23-40 .

enter image description here

The two separate estimations are here enter image description here and here enter image description here and here enter image description here

As to whether or not the segmentation ( discarding the first 22 observations) yields a more accurate prediction of some future values is unknown as it all depends on the origin of the forecast , the length of the forecast and the dictatorial impact of these future values. As they say ,"Results may vary ! " as only your data knows whether or not your analysis has been correctly/sufficiently accomplished. All I know is that one needs to know and test the assumptions of any model imposed on the data.

If one simulates an ar(1) process with a value of .9 and then follows with realizations from a process using -.9 ... we globally conclude using the ensemble that the series is free of auto-correlation which is quite flawed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.