I'm doing multistep forecasts of univariate time series and a wide range of exogenous leading indicator variables are available. Therefore I'm looking for ways to optimally select and/or combine forecasts. Hanson (2010, "Multi-Step Forecast Model Selection") proposes a leave-h-out cross validation criterion which leaves out the observations t-h+1 up to t-h-1 to compute the forecast error of the observation at time t.

The method looks very appealing since leave-one-out CV tends to overfit my data in multistep forecasts. However, I have barely seen any work based on the method. Does anyone have any experience with leave-h-out cross validation? What is your opinion on the method? Which procedure do you prefer to evaluate multistep forecasts?

Any comments are appreciated. Thank you.

  • $\begingroup$ Welcome to cross validated! I have no experience with time series, but here are my 2 ct: I'd choose h so that there is no correlation left. And I'd probably go for training up to t - h only to test prediction at time t, so not using any training data from later time points as I think it highly unusual to have later measurements in "real" prediction (unless its imputation). $\endgroup$ – cbeleites supports Monica May 31 '16 at 6:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.