If your method is in the forecast class and you have more than 24 observations, you should have no problem in running tsCV with h = 24. It will return a matrix with h = 24 columns and the number of rows equals to your data size.
If you only want the cv errors for K=24, then you can do the following: if d is the number of days in your data you, the code
tsCV(data, forecast_method, h = 24)[24*(1:(d-1)),]
should return a error matrix for the whole day predicting only midnight (K=24). The code below illustrates this (d=5)
set.seed(10)
X <- ts(rnorm(5*24))
tsCV(X, rwf, h = 24)[24*(1:4),]
It returns a $4\times 24$ matrix, each row contains the prediction at midnight for some hour of that day, which is given by the column. For instance, element [2,15] is the prediction error at midnight of the second day for hour 15. Then you can apply a metric you see fit to choose the forecast method.
Edit: As you have stated, this is a hack, and quite a slow one actually since it runs 24X more than you actually need. I went to the source code of tsCV and changed it to add a "step" parameter.
The code for the function definition goes below, just copy and run this in a R script. I commented on the lines that I added or modified.
tsCV_step <- function (y, forecastfunction, h = 1, window = NULL, xreg = NULL,
initial = 0, step = 1, ...)
{
y <- as.ts(y)
n <- length(y)
step <- round(step)
step_ind <- seq(step, n - 1L, by = step) ### Added line
e <- ts(matrix(NA_real_, nrow = floor(n/step), ncol = h)) ### Modified line: n was replaced by floor(n/step)
if (initial >= n)
stop("initial period too long")
tsp(e)[-2] <- tsp(y)[-2] ### Modified line: [-2] added
if (!is.null(xreg)) {
xreg <- ts(as.matrix(xreg))
if (NROW(xreg) != length(y))
stop("xreg must be of the same size as y")
tsp(xreg) <- tsp(y)
}
if (is.null(window))
indx <- seq(1 + initial, n - 1L)
else indx <- seq(window + initial, n - 1L, by = 1L)
indx <- intersect(indx, step_ind) ### Added line
for (i in indx) {
y_subset <- subset(y, start = ifelse(is.null(window),
1L, ifelse(i - window >= 0L, i - window + 1L, stop("small window"))),
end = i)
if (is.null(xreg)) {
fc <- try(suppressWarnings(forecastfunction(y_subset,
h = h, ...)), silent = TRUE)
}
else {
xreg_subset <- as.matrix(subset(xreg, start = ifelse(is.null(window),
1L, ifelse(i - window >= 0L, i - window + 1L,
stop("small window")))))
fc <- try(suppressWarnings(forecastfunction(y_subset,
h = h, xreg = xreg_subset, ...)), silent = TRUE)
}
if (!is.element("try-error", class(fc))) {
e[i/step, ] <- y[i + (1:h)] - fc$mean ### Modified Line: i replaced by i/step
### for "e" first index
}
}
if (h == 1) {
return(e[, 1L])
}
else {
colnames(e) <- paste("h=", 1:h, sep = "")
return(e)
}
}
The following example illustrates that it gives the same result as before and is way faster: tsCV_step took approximately 0.4 seconds to compute, while tsCV took more than 10 seconds. The all(step_err == no_step_err, na.rm = T) output shows that the hack output and tsCV_step output is the same, with the only difference that the hack goes to d now instead of (d-1), so the NA's will appear.
set.seed(10)
days = 100
data <- ts(rnorm(24*days))
K_step = 24
# Compute the errors with step and the time taken
init <- Sys.time()
step_err <- tsCV_step(data, rwf, h = 24, step = K_step)
tot_time_step <- Sys.time() - init
init <- Sys.time()
no_step_err <- tsCV(data, rwf, h = 24)[K_step*(1:d),]
tot_time_no_step <- Sys.time() - init
cat(paste("tsCV_step time taken: ", tot_time_step, "\n",
"tsCV time taken: ", tot_time_no_step, sep = ""))
all(step_err == no_step_err, na.rm = T)
Obs1: I'm no expert on R programming, so I might have introduced a bug by changing the source code. It seems stable for your application if you are not using the other parameters like window and xreg.
Obs2: The NA's are also reported, just like in tsCV.
