Forecasting several periods with machine learning

Question

I lately recapped my Time Series knowledge and realised that machine learning mostly gives only one step ahead forecasts.

With one-step-ahead forecasts I mean forecasts which, e.g., if we have hourly data, use the data from 10am to forecast 11am and 11am for 12am etc.

Can machine learning methods produce an h-steps-ahead forecasts? With h-step-ahead forecasts I mean that, e.g., assuming hourly data, we use the data from 10am to make a 7-step-ahead forecast to get estimates for 11,12,13,14,15,16,17 'o clock.

Example pic:

Related to my main question I wonder:

• What are the reasons that I don't see anyone using machine learning to make h-step-ahead forecasts?
• If there is a method using machine learning, is it more or less precise than ARIMA?

Answer 1

(Part of this is taken from a previous post of mine) First of all you need to distinguish the two different ways to perform multistep times series forecasting: Recursive forecasting and direct forecasting:

• In recursive forecasting (also called iterated forecasting) you train your model for one step ahead forecasts only. After the training is done you apply your final model recursively to forecast 1 step ahead, 2 steps ahead, etc...until you reach the desired $n$ steps forecast horizon. To do so, you feed the forecast from each successive step back into the model to generate the next step. This approach is used by traditional forecasting algorithms like ARIMA and Exponential Smoothing algorithms, and can be also used for Machine Learning based forecasting (see this post for an example, and this post for some discussion).
• Direct forecasting is when you train a separate model for each step (so you are trying to "directly" forecast the $n^{th}$ step ahead instead of reaching $n$ steps recursively. See Ben Taied et al. for a discussion of direct forecasting and more complex combined approaches.

Now to answer your main question:

Can machine learning methods produce an h-steps-ahead forecasts?

Yes ML methods can, and they can produce h-steps ahead forecast using both recursive and direct multistep forecasts. Not only that, but for direct multi-step forecasting they are actually more suited to the task than traditional models like ARIMA or Exponential Smoothing. Note however that for direct multi-step forecasting, you need to specify before hand the h-steps ahead for which you want to forecast and train your model accordingly, whereas for recursive forecasting you can use your model for any number of future steps you want.

Moreover Chevillon & Hendry argue that in some cases direct multi-step forecasting is more accurate than recursive forecasting - implying that ML would be more accurate than traditional methods.

For your other questions:

• What are the reasons that I don't see anyone using machine learning to make h-step-ahead forecasts?

Many people are using ML for multi-step forecasting, especially using neural netwroks: Hyndman's nnetar method available in the R Forecast package, Kourentzes' nnfor R package, Amazon's DeepAR model, and many others.

XGBoost has been used successfully in a few Kaggle time series competitions as well.

See Bontempi et al. for a general discussion.

• If there is a method using machine learning, is it more or less precise than ARIMA?

That is an open question, and obviously depends on the data and the application that one is forecasting for.

Answer 2

I have been playing with time series for anomaly detection in the last few months and I can share with you my experience.

The time series I've been working with was characterized by two seasonalities (daily and weekly), no trend and many peaks during daylights.

I did several experiments and then I chose a model based on LSTM neural nets because in my case it outperformed arima, but of course as everything in statistics, there's not an general solution.

To predict more than one time step in the future with a neural net is pretty simple, you will need to output N values instead of one and that N output will be compared to the real N observations.

From my experience I can tell you that by using a low N (say 1), the model will strictly use few time step in the past to predict the new one, without really "learning" the seasonality. On the other side, by increasing N too mutch, the seasonalities are learned but the overall accuracy decreases.

For the purpose of my analysis I found N = 4 (2 hours in the future) to be a good compromise.

Answer 3

To answer your question in a more general term, it is possible to use machine learning and predict h-steps-ahead forecasts. The tricky part is that you have to reshape your data into a matrix in which you have, for each observation the actual value of the observation and past values of the time series for a defined range. You will need to define manually what is the range of data that appear relevant to predict your time series, in fact, as you would parameter an ARIMA model. The width/horizon of the matrix is critical to predict correctly the next value taken by your matrix. If your horizon is restricted, you might miss seasonality effects.

Once you have done that, to predict h-steps-ahead, you will need to predict the first next value based on your last observation. You will then have to store the prediction as an "actual value", which will be used to predict the second next value through a time shifting, just like an ARIMA model. You will have to iterate the process h times to get your h-steps-ahead. Each iteration will rely on the previous prediction.

An example using R code would be the following one.

library(forecast)
library(randomForest)

# create a daily pattern with random variations
myts <- ts(rep(c(5,6,7,8,11,13,14,15,16,15,14,17,13,12,15,13,12,12,11,10,9,8,7,6), 10)*runif(120,0.8,1.2), freq = 24)
myts_forecast <- forecast(myts, h = 24) # predict the time-series using ets + stl techniques
pred1 <- c(myts, myts_forecast1$mean) # store the prediction

# transform these observations into a matrix with the last 24 past values
idx <- c(1:24)
designmat <- data.frame(lapply(idx, function(x) myts[x:(215+x)])) # create a design matrix
colnames(designmat) <- c(paste0("x_",as.character(c(1:23))),"y")

# create a random forest model and predict iteratively each value
rfModel <- randomForest(y ~., designmat)
for (i in 1:24){
  designvec <- data.frame(c(designmat[nrow(designmat), 2:24], 0))
  colnames(designvec) <- colnames(designmat)
  designvec$y <- predict(rfModel, designvec)
  designmat <- rbind(designmat, designvec)
}
pred2 <- designmat$y

#plot to compare predictions
plot(pred1, type = "l")
lines(y = pred2[216:240], x = c(240:264), col = 2)

Now obviously, there are no general rules as to determine whether a time-series model or a machine learning model are more efficient. Computational time may be higher for machine learning models, but on the other hand, you may include any type of additional features to predict your time-series using them (e.g. not just numerical or logical features). General advice would be to test both, and pick the most efficient model.