Using rolling windows to compute out of sample accuracy

Question

I’ve already written this question, but probably I didn’t specified it well, for this reason I write it again. I need to use a random walk model (no-change) yt = yt(1+t) to compute the ratio of RMSFE. What I would like to do is:

1. Fit the model to the data yt,...,yt+k−1 and let yˆt+k be the forecast for the next observation.
2. Compute the forecast error as et=yˆt+k−yt+k.

3. Repeat for t=1,...,n−k"

   residuals1 <- rep(0,58)
   residuals6 <- rep(0,58)
   residuals12 <- rep(0,58)
   
   y1 <- t(y[,1])
   for (i in 1:58) {       
     residuals1[i] <- y1[134+i+1]-y1[134+i]    
     residuals6[i] <- y1[134+i+6]-y1[134+i] 
     residuals12[i] <- y1[134+i+12]-y1[134+i] 
   }
   

Is it a correct way to compute the out.of sample forecasting errors or am I missing something? I would appreciate any suggestions. Thanks!

Answer 1

As you are using a naive (random walk) model, there are no parameters, so it makes no sense to talk of in-sample and out-of-sample, or of training sets and test sets. There is nothing to train.

On the other hand, you may be doing this to compare the results with other methods applied to a training set. I'll assume that's the case.

I'm guessing your data consists of 204 observations (134+58+12) and you are using the first 135 as a training set. There's no need for loops here as the computation is easily vectorized. Also, because you use a loop you ignore some of the available forecast errors in the test set.

I think you can get the results you want as follows:

y <- rnorm(204)
residuals1 <- diff(y[135:204],lag=1)
residuals6 <- diff(y[135:204],lag=6)
residuals12 <- diff(y[135:204],lag=12)