# forecasting with optimised theta method (otm) using time series cross validation with R

I want to do an out-of-sample forecast experiment using the optimised theta method (otm) on a time series. Further, time series cross validation with a fixed rolling window size should be applied. Therefore i tried to combine the tscv function with the forecTheta package:

## [Updated]
library(forecTheta)
library(forecast)
library(tseries)

#y is the time series
y1 = 2+ 0.15*(1:20) + rnorm(20,2)
y2 = y1[20]+ 0.3*(1:30) + rnorm(30,2)
y =  as.ts(c(y1,y2))

#10 obs in the test set, and 40 obs in the training set
ntest <- 10
ntrain <- length(y)-ntest

# create a function with optimised theta method
# theta is within the bounds [1,5]
ftheta <- function(x,h){otm.arxiv(x,h=h,thetaList=seq(from=1,to=5,by=0.5),g="SE")}

# in each rolling window 40 obs are included
e <- tsCV(y,ftheta,h = 3,window=40)



However when i try to run this code the e1 vector, which includes the forecast errors, only has NA values. First I thought this would be due to no limit in the theta parameter, therefore i restricted this parameter. But this diddnt help. Also i tried to implement/execute the forecast in the forecTheta package, but it diddnt work. [solved]

I have another question regarding the predicted values. For the 1-step ahead forecast the predicted values are obtained with the following:

#predicted values for h=1
fc1 <- c(NA,y[2:50]-e[1:49,1])
fc1 <- fc1[41:50]


However I´m curious whether the predicted values for the 3-step ahead are coded correctly. Since the first 3-step ahead forecast is the prediction of the 43th observation, i guess the following line is wrong:

fc3 <- c(NA,y[2:50]-e[1:49,3])
fc3 <- fc3[41:50]


There are multiple issues present.

1. The first one is that otm.arxiv() does not follow the standard R practice of returning a fitted model that one applies forecast() to. Instead, it performs fitting and forecasting. To obtain a forecast from otm.arxiv(), you need to supply an h parameter to it:

otm.arxiv(y,h=3,thetaList=seq(from=1,to=5,by=0.5),g="SE")

2. The second problem is that otm.arxiv() will apparently throw an error if h<3: whereas the previous line gives us a forecast, changing it to h=2

otm.arxiv(y,h=2,thetaList=seq(from=1,to=5,by=0.5),g="SE")


gives us the rather unhelpful error message

Error in groe function: m<1


(Incidentally, this is not due to your control parameters; we get the same error for otm.arxiv(y,h=2).)

You may want to bring this to the package maintainers' attention. At the very least, it would be good to see the restriction on h in the help page, and to have this error caught with a more informative error message.

So overall, you can get a useful e1 error matrix by changing two lines as follows:

ftheta <- function(x,h){otm.arxiv(x,h=h,thetaList=seq(from=1,to=5,by=0.5),g="SE")}
e1 <- tsCV(y,ftheta,h = 3,window=40)


Of course, e1 is now a 50 by 3 time series matrix, not a vector of length 50 any more, so you will need to adapt the rest of your code. And of course it still contains some NA values for the initialization part and where we can't forecast the full 3 periods at the end.

• Thanks a lot for your input! It works perfectly. Just a question for my understanding: when i change h in the e1 code line to h=6, i get different errors for the h=3 forecast, compared to when i use h=3 in the e1 code. Why is that?
– 29ML
Jun 16 '20 at 7:52
• That is a very good question. I would expect that there is something in the OTM algorithm where the fitting depends on the forecast horizon, so that supplying h=6 yields a different model than h=3. (This would be a reason for the first point I raise, which is a nonstandard design decision.) Unfortunately, I have no understanding of the OTM, so I can't help you there. Sorry! Jun 16 '20 at 8:00
• Good point! Sounds reasonable to me. Thanks again for your help.
– 29ML
Jun 16 '20 at 8:13