0
$\begingroup$

My training set has 13,737 observations with 53 predictors. I need to compare the accuracy of Random Forest vs. GBM.

For Random Forest, I set ntree = 128 [based on Oshiro et al. (2012)] in train(data=trainset, y~., method = "rf", ntree = 128) because the default (500) was taking far too long. At 128 trees, it took 1.25 hours.

Now in train(data=trainset, y~., method = "gbm", verbose = FALSE), I have not changed the default n.tree value (100). It has been running for 2 hour now...

Should I set n.tree in gbm also to 128?
Would it be wrong to compare it against random forest otherwise?

Please advise.

Thank you!

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ I can't think of any reason why the same number of trees should be optimal in both methods. Bagging and boosting work quite differently. $\endgroup$
    – mkt
    Commented Feb 19, 2019 at 16:43
  • $\begingroup$ For the random forest, you could take away a lot of the pain by using a smaller value of sampsize such as sampsize=1000. At the moment, each bootstrap sample of your data is of size 8681, which is the main reason why it is taking so long. $\endgroup$
    – Flounderer
    Commented Feb 19, 2019 at 18:42
  • $\begingroup$ Isn't the default setting for sampsize (at least in randomForest package of R) equal to the size of the dataset? I.E. 13,737 not 8,681. $\endgroup$
    – astel
    Commented Feb 19, 2019 at 18:58
  • $\begingroup$ @astel I believe that's only the default if you specify replace=T but it might be the other way round. $\endgroup$
    – Flounderer
    Commented Feb 19, 2019 at 20:27
  • $\begingroup$ Right, though replace = T is also the default $\endgroup$
    – astel
    Commented Feb 19, 2019 at 20:31

1 Answer 1

Reset to default
3
$\begingroup$

The number of trees does not need to be the same for each algorithm in order for a comparison to be drawn. What you should do is optimize the number of trees separately for each algorithm through some sort of grid search using cross validation (though it looks like you don't have the time for this as it can be a lengthy process). After optimizing it would be highly unlikely that the number of trees that optimize each algorithm is the same.

Improve this answer
$\endgroup$
2
  • $\begingroup$ ... noting that for RF, more trees are always better, so it's more a matter of determining how many trees you can run in the allotted time, whereas for GBM, more trees (after a point) lead to more overfitting. $\endgroup$
    – jbowman
    Commented Feb 19, 2019 at 18:11
  • $\begingroup$ Thank you @astel! I did attempt to trim the number of trees for Random Forest trying 2^x trees at a time. And found that I only needed 64 for near-perfect accuracy, not even 128. And then looking at the finalModel plot, it seems class errors flattened out at around 45 trees only, not declining much after. GBM - this gave me trouble when I tried changing n.tree from the default. So just left it as is. RF is working better for the data. $\endgroup$
    – enkay
    Commented Feb 20, 2019 at 7:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.