I have a question related to modeling short time-series. It is not a question if to model them, but how. What method would you recommend for modeling (very) short time-series (say of length $T \leq 20$)? By "best" I mean here the most robust one, that is the least prone to errors due the fact of limited numbers of observations. With short series single observations could influence the forecast, so the method should provide a cautious estimate of errors and possible variability connected to the forecast. I am generally interested in univariate time-series but it would be also interesting to know about other methods.
$\begingroup$
$\endgroup$
7
6 Answers
$\begingroup$
$\endgroup$
0
It is very common for extremely simple forecasting methods like "forecast the historical average" to outperform more complex methods. This is even more likely for short time series. Yes, in principle you can fit an ARIMA or even more complex model to 20 or fewer observations, but you will be rather likely to overfit and get very bad forecasts.
So: start with a simple benchmark, e.g.,
- the historical mean
- the historical median for added robustness
- the random walk (forecast the last observation out)
Assess these on out-of-sample data. Compare any more complex model to these benchmarks. You may be surprised at seeing how hard it is to outperform these simple methods. In addition, compare the robustness of different methods to these simple ones, e.g., by not only assessing average accuracy out-of-sample, but also the error variance, using your favorite error measure.
Yes, as Rob Hyndman writes in his post that Aleksandr links to, out-of-sample testing is a problem in itself for short series - but there really is no good alternative. (Don't use in-sample fit, which is no guide to forecasting accuracy.) The AIC won't help you with the median and the random walk. However, you could use time-series cross-validation, which AIC approximates, anyway.
answered Jan 27, 2015 at 11:37
$\begingroup$
$\endgroup$
6
I am again using a question as an opportunity to learn more about time series - one of the (many) topics of my interest. After a brief research, it seems to me that there exist several approaches to the problem of modeling short time series.
The first approach is to use standard/linear time series models (AR, MA, ARMA, etc.), but to pay attention to certain parameters, as described in this post [1] by Rob Hyndman, who does not need an introduction in time series and forecasting world. The second approach, referred to by most of the related literature that I have seen, suggest using non-linear time series models, in particular, the threshold models [2], which include threshold autoregressive model (TAR), self-exiting TAR (SETAR), threshold autoregressive moving average model (TARMA), and TARMAX model, which extends TAR model to exogenous time series. Excellent overviews of the non-linear time series models, including threshold models, can be found in this paper [3] and this paper [4].
Finally, another IMHO related research paper [5] describes an interesting approach, which is based on Volterra-Weiner representation of non-linear systems - see this [6] and this [7]. This approach is argued to be superior to other techniques in the context of short and noisy time series.
References
- Hyndman, R. (March 4, 2014). Fitting models to short time series. [Blog post]. Retrieved from http://robjhyndman.com/hyndsight/short-time-series
- Pennsylvania State University. (2015). Threshold models. [Online course materials]. STAT 510, Applied Time Series Analysis. Retrieved from https://online.stat.psu.edu/stat510/lesson/13/13.2
- Zivot, E. (2006). Non-linear time series models. [Class notes]. ECON 584, Time Series Econometrics. Washington University. Retrieved from http://faculty.washington.edu/ezivot/econ584/notes/nonlinear.pdf
- Chen, C. W. S., So, M. K. P., & Liu, F.-C. (2011). A review of threshold time series models in finance. Statistics and Its Interface, 4, 167–181. Retrieved from http://intlpress.com/site/pub/files/_fulltext/journals/sii/2011/0004/0002/SII-2011-0004-0002-a012.pdf
- Barahona, M., & Poon, C.-S. (1996). Detection of nonlinear dynamics of short, noisy time series. Nature, 381, 215-217. Retrieved from http://www.bg.ic.ac.uk/research/m.barahona/nonlin_detec_nature.PDF
- Franz, M. O. (2011). Volterra and Wiener series. Scholarpedia, 6(10):11307. Retrieved from http://www.scholarpedia.org/article/Volterra_and_Wiener_series
- Franz, M. O., & Scholkopf, B. (n.d.). A unifying view of Wiener and Volterra theory and polynomial kernel regression. Retrieved from http://www.is.tuebingen.mpg.de/fileadmin/user_upload/files/publications/nc05_%5B0%5D.pdf
answered Jan 26, 2015 at 22:43
-
2$\begingroup$ +1 for the link to Rob Hyndman's post. (However, I am tempted to -1 for the complex models. I'd be extremely wary of using threshold or any other nonlinear time series methods on time series of less than 20 observations. You are almost certain to overfit, which goes directly counter to the OP's requirement of a robust method.) $\endgroup$ Commented Jan 27, 2015 at 11:25
-
$\begingroup$ @StephanKolassa: Thank you for the comment and +1. I understand (mostly intuitively) your strong doubts on using complex models for short time series. However, I'm not sure then how to explain the presence of such suggestions in the peer-reviewed literature I've referred to. Any clarifications will be appreciated. $\endgroup$ Commented Jan 27, 2015 at 11:43
-
4$\begingroup$ [2,3,4] do not mention short time series, and look at the plots in [2]: >120 observations. [4] concentrates on finance, where you have enormously more than 20 observations. [5] writes about "short time series, typically 1,000 points long" (p. 216). I see no way to reliably and robustly fit a TAR or similar model, or any of the more complex ones you link to, with <20 observations. (BTW: I also do some inferential statistics on the side, and with fewer than 20 observations, you really can't estimate more than the mean and one more parameter.) $\endgroup$ Commented Jan 27, 2015 at 11:55
-
$\begingroup$ I see. In regard to [2,3,4], you're right that those papers don't mention short TS - I guess, I read in some other sources about possibility of applying TAR family of models to short TS and then, mentally connecting the topics, found and provided the references in question as better overviews of TAR models. I appreciate the clarification. $\endgroup$ Commented Jan 27, 2015 at 12:08
-
6$\begingroup$ You're welcome ;-) I guess the takeaway is that "short" is very context-dependent: for sensor reading series or in finance, 1000 data points is "short" - but in supply chain management, 20 monthly observations is almost normal, and "short" will only start at 12 or fewer observations. $\endgroup$ Commented Jan 27, 2015 at 12:15
$\begingroup$
$\endgroup$
1
No, There is no best univariate extrapolation method for a short time series with $T \leq 20$ series. Extrapolation methods need lots and lots of data.
Following qualitative methods work well in practice for very short or no data:
- Composite forecasts
- Surveys
- Delphi method
- Scenario building
- Forecast by analogy
- Executive opinion
One of the best methods that I know that works very well is the use of structured analogies (5th in the list above) where you look for similar/analogous products in the category that you are trying to forecast and use them to forecast short term forecasting. See this article for examples, and SAS paper on "how to" do this using of course SAS. One limitation is that forecast by analogies will work only of you have good analogies otherwise you could rely on judgemental forecasting. Here is another video from Forecastpro software on how to use a tool like Forecastpro to do forecasting by analogy. Choosing an analogy is more art than science and you need domain expertise to select analogous products/situations.
Two excellent resources for short or new product forecasting:
- Principle of Forecasting by Armstrong
- New Product forecasting by Kahn
The following is for illustrative purpose.I just finished reading Signal and Noise by Nate Silver, in that there is a good example on US and Japanese(analogue to US market) housing market bubble and prediction. In the chart below if you stop at 10 data points and use one of the extrapolation methods (exponential smooting/ets/arima...) and see where it takes you and where the actual ended. Again the example I presented is much more complex than simple trend extrapolation. This is just to highlight the risks of trend extrapolation using limited data points. In addition if your product has seasonal pattern, you have to use some form of analogous products situation to forecast. I read an article I think in Journal of Business research that if you have 13 week of product sales in pharmaceuticals, you could predict data with greater accuracy using analogous products.
answered Jan 26, 2015 at 22:45
-
$\begingroup$ Thanks for pointing out a different approach! And I agree, Nate Silvers book is great. $\endgroup$– TimCommented Jan 27, 2015 at 17:09
$\begingroup$
$\endgroup$
The assumption that the number of observations is critical came from an off-handed comment by G.E.P. Box regarding the minimum sample size to identify a model. A more nuanced answer as far as I am concerned is that the problem/quality of model identification is not solely based upon the sample size but the ratio of signal to noise that is in the data. If you have a strong signal to noise ratio you need less observations. If you have low s/n then you need more samples to identify. If your data set is monthly and you have 20 values it is not possible to empirically identify a seasonal model HOWEVER if you think the data might be seasonal then you could start the modelling process by specifying an ar(12) and then do model diagnostics (tests of significance) to either reduce or to augment your structurally deficient model
$\begingroup$
$\endgroup$
With very limited data, I would be more inclined to fit the data using Bayesian techniques.
Stationarity can be a bit tricky when dealing with Bayesian time series models. One choice is to enforce constraints on parameters. Or, you could not. This is fine if you just want to look at the distribution of the parameters. However, if you want to generate the posterior predictive, then you might have a lot of forecasts that explode.
The Stan documentation provides a few examples where they put constraints on the parameters of time series models to ensure stationarity. This is possible for the relatively simple models they use, but it can be pretty much impossible in more complicated time series models. If you really wanted to enforce stationarity, you could use a Metropolis-Hastings algorithm and throw out any coefficients that are improper. However, this requires a lot of eigenvalues to be calculated, which will slow things down.
$\begingroup$
$\endgroup$
The problem as you wisely pointed out is the "overfitting" caused by fixed list-based procedures. A smart way is to try and keep the equation simple when you have a negligible amount of data. I have found after many moons that if you simply use an AR(1) model and leave the rate of adaption ( the ar coefficient) to the data things can work out reasonably well. For example if the estimated ar coefficient is close to zero this means that the overall mean would be appropriate . if the coefficient is near +1.0 then this means that the last value (adjusted for a constant is more appropriate . If the coefficient is close to -1.0 then the negative of the last value (adjusted for a constant) would be the best forecast. If the coefficient is otherwise it means that a weighted average of the recent past is appropriate.
This is precisely what AUTOBOX starts with and then discards anomalies as it fine tunes the estimated parameter when a "small # of observations" is encountered.
This is an example of the "art of forecasting" when a pure data driven approach might be inapplicable.
Following is an automatic model developed for the 12 data points without concern for anomalies. 
with Actual/Fit and Forecast here 
and residual plot here 
Mcomppackage for R), 504 have 20 or fewer observations, specifically 55% of the yearly series. So you could look up the original publication and see what worked well for yearly data. Or even dig through the original forecasts submitted to the M3 competition, which are available in theMcomppackage (listM3Forecast). $\endgroup$