0
$\begingroup$

My training set has 13,737 observations with 53 predictors. I need to compare the accuracy of Random Forest vs. GBM.

For Random Forest, I set ntree = 128 [based on Oshiro et al. (2012)] in train(data=trainset, y~., method = "rf", ntree = 128) because the default (500) was taking far too long. At 128 trees, it took 1.25 hours.

Now in train(data=trainset, y~., method = "gbm", verbose = FALSE), I have not changed the default n.tree value (100). It has been running for 2 hour now...

Should I set n.tree in gbm also to 128?
Would it be wrong to compare it against random forest otherwise?

Please advise.

Thank you!

$\endgroup$
  • 1
    $\begingroup$ I can't think of any reason why the same number of trees should be optimal in both methods. Bagging and boosting work quite differently. $\endgroup$ – mkt - Reinstate Monica Feb 19 '19 at 16:43
  • $\begingroup$ For the random forest, you could take away a lot of the pain by using a smaller value of sampsize such as sampsize=1000. At the moment, each bootstrap sample of your data is of size 8681, which is the main reason why it is taking so long. $\endgroup$ – Flounderer Feb 19 '19 at 18:42
  • $\begingroup$ Isn't the default setting for sampsize (at least in randomForest package of R) equal to the size of the dataset? I.E. 13,737 not 8,681. $\endgroup$ – astel Feb 19 '19 at 18:58
  • $\begingroup$ @astel I believe that's only the default if you specify replace=T but it might be the other way round. $\endgroup$ – Flounderer Feb 19 '19 at 20:27
  • $\begingroup$ Right, though replace = T is also the default $\endgroup$ – astel Feb 19 '19 at 20:31
3
$\begingroup$

The number of trees does not need to be the same for each algorithm in order for a comparison to be drawn. What you should do is optimize the number of trees separately for each algorithm through some sort of grid search using cross validation (though it looks like you don't have the time for this as it can be a lengthy process). After optimizing it would be highly unlikely that the number of trees that optimize each algorithm is the same.

$\endgroup$
  • $\begingroup$ ... noting that for RF, more trees are always better, so it's more a matter of determining how many trees you can run in the allotted time, whereas for GBM, more trees (after a point) lead to more overfitting. $\endgroup$ – jbowman Feb 19 '19 at 18:11
  • $\begingroup$ Thank you @astel! I did attempt to trim the number of trees for Random Forest trying 2^x trees at a time. And found that I only needed 64 for near-perfect accuracy, not even 128. And then looking at the finalModel plot, it seems class errors flattened out at around 45 trees only, not declining much after. GBM - this gave me trouble when I tried changing n.tree from the default. So just left it as is. RF is working better for the data. $\endgroup$ – enkay Feb 20 '19 at 7:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.