# why isn't the tscv function allowing for step-size other than 1?

the tsCV function from the forecast package allows for forecasting using a rolling forecast window. Why is it that the step-size is always 1 sample ahead, and the function doesn't allow for stepping k samples for each forecast?

My business scenario is that I have to predict 24 samples ahead (h=24), but I'm forecasting only at midnight of each day (k=24), it's not that I have to predict 24 hours ahead every hour. I would like to simulate that using tsCV.

Note that in the sklearn counterpart TimeSeriesSplit this functionality is implemented, and the difference between successive training sets isn't hard-coded to 1 sample.

If your method is in the forecast class and you have more than 24 observations, you should have no problem in running tsCV with h = 24. It will return a matrix with h = 24 columns and the number of rows equals to your data size.

If you only want the cv errors for K=24, then you can do the following: if d is the number of days in your data you, the code

tsCV(data, forecast_method, h = 24)[24*(1:(d-1)),]


should return a error matrix for the whole day predicting only midnight (K=24). The code below illustrates this (d=5)

set.seed(10)
X <- ts(rnorm(5*24))
tsCV(X, rwf, h = 24)[24*(1:4),]


It returns a $$4\times 24$$ matrix, each row contains the prediction at midnight for some hour of that day, which is given by the column. For instance, element [2,15] is the prediction error at midnight of the second day for hour 15. Then you can apply a metric you see fit to choose the forecast method.

Edit: As you have stated, this is a hack, and quite a slow one actually since it runs 24X more than you actually need. I went to the source code of tsCV and changed it to add a "step" parameter.

The code for the function definition goes below, just copy and run this in a R script. I commented on the lines that I added or modified.

tsCV_step <- function (y, forecastfunction, h = 1, window = NULL, xreg = NULL,
initial = 0, step = 1, ...)
{
y <- as.ts(y)
n <- length(y)
step <- round(step)
step_ind <- seq(step, n - 1L, by = step) ### Added line

e <- ts(matrix(NA_real_, nrow = floor(n/step), ncol = h)) ### Modified line: n was replaced by floor(n/step)
if (initial >= n)
stop("initial period too long")
tsp(e)[-2] <- tsp(y)[-2] ### Modified line: [-2] added
if (!is.null(xreg)) {
xreg <- ts(as.matrix(xreg))
if (NROW(xreg) != length(y))
stop("xreg must be of the same size as y")
tsp(xreg) <- tsp(y)
}
if (is.null(window))
indx <- seq(1 + initial, n - 1L)
else indx <- seq(window + initial, n - 1L, by = 1L)
indx <- intersect(indx, step_ind) ### Added line
for (i in indx) {
y_subset <- subset(y, start = ifelse(is.null(window),
1L, ifelse(i - window >= 0L, i - window + 1L, stop("small window"))),
end = i)
if (is.null(xreg)) {
fc <- try(suppressWarnings(forecastfunction(y_subset,
h = h, ...)), silent = TRUE)
}
else {
xreg_subset <- as.matrix(subset(xreg, start = ifelse(is.null(window),
1L, ifelse(i - window >= 0L, i - window + 1L,
stop("small window")))))
fc <- try(suppressWarnings(forecastfunction(y_subset,
h = h, xreg = xreg_subset, ...)), silent = TRUE)
}
if (!is.element("try-error", class(fc))) {
e[i/step, ] <- y[i + (1:h)] - fc$mean ### Modified Line: i replaced by i/step ### for "e" first index } } if (h == 1) { return(e[, 1L]) } else { colnames(e) <- paste("h=", 1:h, sep = "") return(e) } }  The following example illustrates that it gives the same result as before and is way faster: tsCV_step took approximately 0.4 seconds to compute, while tsCV took more than 10 seconds. The all(step_err == no_step_err, na.rm = T) output shows that the hack output and tsCV_step output is the same, with the only difference that the hack goes to d now instead of (d-1), so the NA's will appear. set.seed(10) days = 100 data <- ts(rnorm(24*days)) K_step = 24 # Compute the errors with step and the time taken init <- Sys.time() step_err <- tsCV_step(data, rwf, h = 24, step = K_step) tot_time_step <- Sys.time() - init init <- Sys.time() no_step_err <- tsCV(data, rwf, h = 24)[K_step*(1:d),] tot_time_no_step <- Sys.time() - init cat(paste("tsCV_step time taken: ", tot_time_step, "\n", "tsCV time taken: ", tot_time_no_step, sep = "")) all(step_err == no_step_err, na.rm = T)  Obs1: I'm no expert on R programming, so I might have introduced a bug by changing the source code. It seems stable for your application if you are not using the other parameters like window and xreg. Obs2: The NA's are also reported, just like in tsCV. • I did something similar: ret <- ret[seq(attributes(ret)$dim[1]) %% 24 == 1,] - but this is a hack, as the tsCV runs X24 times than necessary, right? (it's very slow when using a complicated forecast...) – ihadanny Oct 15 '19 at 14:02
• Yes, I will try to think how to overcome that problem. I will edit my answer to clarify that! – Lucas Prates Oct 15 '19 at 15:18
• wow thanks!!! checking if there's a simpler solution before accepting – ihadanny Oct 15 '19 at 17:48