How much time will xgboost model take?

Question

I have run xg boost model on R. Dataset has 130 columns and 130,000 observations. I have run xgboost model with default parameters, like this-

xgb_model <- train(x = trainX, y = xtrain$loss, method = "xgbTree")

It has been running for 2 hours. How much time will it take generally? Is it so complex? Can someone explain me in lucid way ,what exactly happens inside xgboost algo ?

Answer 1

It's hard to say. It also depends on the machine you're running it on. You can set a max.depth parameter for the trees. This should speed it up, but you'll lose expressive power, i.e. the accuracy could suffer. Default is set to 6.

The learning rate $\eta \in [0,1]$ (eta) can also speed things up. Low eta value means the model is more robust to over fitting but is slower to compute. Default is set to 0.3.

Try

xgb_model <- train(x = trainX, y = xtrain$loss, method = "xgbTree", eta=1, max.depth=2)

And as Dex Groves said:

You could also change the hyperparameters colsample_bytree and/or colsample_bylevel to something less than 1.0

Answer 2

This question is not answerable. It depends on what is your computational resources. xgboost is good at taking advantages of all the resources you have. And it can run in clusters with hundreds of CPUs. Of course, time would be different for different platforms.

However, you can estimate how long it will take on your computer. Just pay attention to nround, i.e., number of iterations in boosting, the current progress and the target value. For example, if you are seeing 1 minute for 1 iteration (building 1 iteration usually take much less time that you can track), then 300 iterations will take 300 minutes.

To understand boosting and number of iterations you may find following posts helpful.

How does linear base leaner works in boosting? And how it works in xgboost library?

Boosting: why is the learning rate called a regularization parameter?

How to know if a learning curve from SVM model suffers from bias or variance?