XGBoost one-step ahead forecast

Question

I have trained and cross-validated an xgboost classification algorithm in R using the following code:

xgb_params <- list("objective" = "binary:logistic", 
                   "eval_metric" = "error",
                   min_child_weight=1, 
                   subsample=1, 
                   colsample_bytree= 0.6, 
                   eta = 0.05, 
                   gamma = 1, 
                   max_depth = 5
)
watchlist <- list(train = train_matrix, test = test_matrix) 
xgb_mod <- xgb.train(params = xgb_params, 
                     data = train_matrix,
                     nrounds = 800,
                     watchlist = watchlist, 
                     seed = 333)

xgb_mod

Now I want to do one-step-ahead forecasting.

However, using the following:

xgbpred_prob <- predict(xgb_mod, newdata = test_matrix)

it is required some new data to be stored into a matrix. Instead, I wish to do forecasting like the following code would do for an ARIMA model:

fit <- arima(df, order = c(0,1,1)) 
predict(fit, n.ahead = 6)

It is like if the first part of the job which I have done was to validate the booster, instead now I wish to put the model into production mode and use it on a daily basis for daily forecasting.

Do you have any idea how could I achieve that?

Answer 1

It depends on what you have in the train_matrix data.

if you only have time variables (minutes, day, month, is_weekend, .. ), you can generate new features for future data and use it for prediction. I believe this is the case for df in the arima example.

But if you have other features (like weather data), you can't predict future data without also providing those features.

Answer 2

it is required some new data to be stored into a matrix

Yes, for models like xgboost and others you have to put the time features into a matrix/dataframe.

If the goal is for forecasting something like a time series, then the features must be time related. One way to do this is to have columns for things like

• Year
• Month
• Week of Year
• Hour
• Minute

Etc, to whatever granularity you require. In order to predict on new data, your new observations need to be in the same format as your training data. When your features are time based, this is pretty easy. For instance, if today you are predicting for tomorrow you already know tomorrows day, month, year, etc etc.

As Hicham Moad Safhi points out, whatever you use to train your model needs to be available at prediction time. So, if you use the weather to predict something, you're going to need to know the weather for tomorrow today. Not so bad, we have weather forecasts and the like, but that isn't true of all data.