4
$\begingroup$

Following up on a discussion in a previous thread:

Are there any situations where adding too much data to a forecasting model is counterproductive, in the sense that it reduces the forecasting accuracy of the model?

Specifically, if I have a univariate time series model, is there such thing as using too much history? Are there situations where I have 20 years of monthly data, but I would be better off using only the most recent 5 years of data, since older data would reduce the accuracy of the model.

For example if there was a structural break in the time series, does feeding data from before the break to the time series somehow throw the model off?

Or does the ML truism that "more data = better accuracy" hold for time series as well?

Improve this question
$\endgroup$
4
  • $\begingroup$ As always, it depends. Often some pieces of data are of lower quality than others: throwing lots of low-quality data at a problem does not necessarily improve the solution. $\endgroup$
    – whuber ♦
    Apr 24, 2018 at 16:27
  • $\begingroup$ Do you mean adding new variables or increasing the sample size of the existing variables? $\endgroup$
    – Richard Hardy
    Apr 24, 2018 at 17:42
  • $\begingroup$ @RichardHardy increasing the sample size. See edit. $\endgroup$
    – Skander H.
    Apr 24, 2018 at 17:48
  • $\begingroup$ @RichardHardy stats.stackexchange.com/questions/343848/… $\endgroup$
    – Skander H.
    May 1, 2018 at 17:46

2 Answers 2

Reset to default
4
$\begingroup$

If the time series data are coming from an unchanging data generating process (DGP), having a longer time series (larger sample size) can only be beneficial. Most models will be fit with greater accuracy on more data and thus will generate better forecasts. (A counterexample would be a spurious regression model for a pair of random walks. There, having a larger sample will not necessarily lead to a better estimate of the slope that is actually zero. Hence, if the model is terrible to begin with, a longer sample might not help in improving it.)

If the time series are coming from a DGP that is evolving over time and the model does not account for that, more data may either be beneficial or detrimental. A longer history may be a more irrelevant history to the extent that it would harm forecasting performance. This depends on how quickly the DGP is changing and how much history you already have, and how much more history you are adding.

[D]oes feeding data from before the break to the time series somehow throw the model off?

Yes, it very well may, unless the break is minuscule and the historical data is relevant enough for the new period.

Improve this answer
$\endgroup$
1
  • $\begingroup$ +1 A formal approach that considers whether adding observations prior to a structural change improves the forecasts or not is: Pesaran MH, Timmermann A (2002). "Market Timing and Return Prediction under Model Instability." Journal of Empirical Finance, 9, 495–510. doi:10.1016/s0927-5398(02)00007-5 $\endgroup$
    – Achim Zeileis
    Apr 28, 2018 at 10:49
1
$\begingroup$

Thanks for the question as it leads to a teaching moment ....

An often overlooked caveat when dealing with data is the assumption that the parameters to be optimized are invariant . In practice with time series data , the question should be asked "have the parameters changed over time" or equivalently do we have too much data ?. Tong in 1980 considered this issue

enter image description here

with the following idea in mind

enter image description here.

Following is a data set from his work suggesting how to deal with this issue.

enter image description here

with acf here

enter image description here

If one identified a model based upon the 40 values, we get

enter image description here

and a residual plot of

enter image description here

The residual plot suggests model deficiency which might be treated by incorporating variance change detection ala Tsay: Outliers, Level Shifts, and Variance Changes in Time Series or more simply parameter change detection via a sequential use of the Chow test for constant parameters where the grouping is based upon a search.

The acf of the residuals does not warn us about the violation.

enter image description here

If we estimate the AR(1) model separately for different possible grouping, we get

enter image description here

suggesting the optimal partitioning is 1-22 vs 23-40.

enter image description here

The two separate estimations are here

enter image description here

and here

enter image description here

As to whether or not the segmentation (discarding the first 22 observations) yields a more accurate prediction of some future values is unknown as it all depends on the origin of the forecast, the length of the forecast and the dictatorial impact of these future values. As they say, “Results may vary!“ as only your data knows whether or not your analysis has been correctly/sufficiently accomplished. All I know is that one needs to know and test the assumptions of any model imposed on the data.

If one simulates an ar(1) process with a value of .9 and then follows with realizations from a process using -.9 ... we globally conclude using the ensemble that the series is free of auto-correlation which is quite flawed.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.