I've been playing around with random forest algorithm to classify a binary Y vector using classification and regression trees. Classification trees output class probability and regression trees an average number between 0 and 1. By setting a threshold of 0.5 for either probability (classification trees) or average (regression trees) I get very similar performance in terms of classification accuracy and similar ranking of feature importance. My question is, is it OK to use regression trees for classification tasks? If so, do you have any reference or example of such use?

  • $\begingroup$ It depends. One the one hand, it depends on the flavour of RFs you're using, in particular the splitting criterion and the aggregation of how individual tree estimates (decisions) are combined to a Random Forest estimate. I believe CART can be used for both (as it's name literally implies). On the other hand, it depends on your definition of "okay". Some theoretical analysis is only valid for the regression setting. $\endgroup$
    – ngmir
    Apr 25, 2023 at 9:05


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.