I have a yearly time series data, from 1980 to 2005. The data is splitted into an training sample and a out of sample; the out-of sample consists of the 6 most recent observations and the rest is considered for training sample. I need to fit a ETS model and compare different accuracy measures for different forecast step aheads h=1,2,3,4,5 and 6.

Something like this:

      h=1  h=2  h=3  h=4  h=5  h=6  ...
MSE   ..   ..   ..   ..   ..   ..   ...
MASE  ..   ..   ..   ..   ..   ..   ...

The following code gives me the accuracy measures for h=6:

trainx<- window(x,end=1999.99)
testx<- window(x,start=2000)
fit<- ets(trainx)

The questions are:

  1. How can I calculate the accuracy measures for h=1,2,3,4,5 ? For instance, when h=2, I fit a model to training data and I produce the forecast that correspond to 2000 and 2001.

  2. Now, how should I produce the forecast for 2002 and 2003, etc?
    Should I suppose that the observations for the year 2000 an 2001 are known and then fit a new model (this time I need to add the observations of 2000 and 2001 to the training set), then, to produce the forecast for 2002 and 2003?


1 Answer 1


It sounds like you need a rolling forecast origin (aka time series cross-validation). Here is an example.

x <- ts(cumsum(rnorm(26)), start=1980)

k <- 10 # minimum data length for fitting a model
n <- length(x)
mae <- matrix(NA,n-k-1,6)
st <- tsp(x)[1] + k - 1

for(i in 1:(n-k-1))
  trainx <- window(x, end=st+i-1)
  testx <- window(x, start=st+i, end=st+i+5)
  fit <- ets(trainx)
  fcast <- forecast(fit, h=6)
  mae[i,1:length(testx)] <- abs(fcast[['mean']]-testx)
mase <- mae / mean(abs(diff(x)))
tab <- rbind(colMeans(mae,na.rm=TRUE),colMeans(mase,na.rm=TRUE))
rownames(tab) <- c("MAE","MASE")
colnames(tab) <- paste("h=",1:6,sep="")

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.