Despite the resembling and other increasing data variability approaches, can the random forest "as an algorithm" be considered a good option for the unbalanced data classification?

  • $\begingroup$ No. (Please be more specific in your question, as it stands it is too broad. You need to clarify your situation as well as what you mean by the statement "as an algorithm" - as opposed to something else?) $\endgroup$ – usεr11852 Oct 27 '16 at 21:40
  • 6
    $\begingroup$ @usεr11852 I don't think it's too broad at all -- it just has a one-word answer. $\endgroup$ – shadowtalker Oct 27 '16 at 23:34
  • $\begingroup$ What I meant by as an algorithm is compared to other classification tools such as SVM, logistic regression,....is RF considered a good option? $\endgroup$ – mhdella Oct 28 '16 at 8:20
  • $\begingroup$ Why not edit your question to show what comparators you are considering and what situation you envisage using your chosen method in? $\endgroup$ – mdewey Oct 28 '16 at 10:44
  • $\begingroup$ @ssdecontrol: I am all for succinct answers; I rarely found one-word answers to be very enlightening though. Your own answer is a proof of that (as it is not one-worded :D ). $\endgroup$ – usεr11852 Oct 29 '16 at 10:38

It's not a good option.

Random forests are built on decision trees, and decision trees are sensitive to class imbalance. Each tree is built on a bag, and each bag is a uniform random sample from the data (with replacement). Therefore each tree will be biased in the same direction and magnitude (on average) by class imbalance.

Several techniques for reducing or mitigating class imbalance exist, some of which are general and some of which are specific to random forests. That topic has been discussed extensively both here and elsewhere.

edit: I would add that I don't think it's dramatically worse than any other option, e.g. logistic regression, although I have no evidence for it

  • $\begingroup$ increase the size of bootstrap samples.. so that on get both the class in every sample. $\endgroup$ – Arpit Sisodia Nov 17 '17 at 8:09
  • $\begingroup$ @ArpitSisodia that will still result in unbalanced samples. You would have to use sampling weights to oversample the rarer class in each bootstrap sample before constructing the tree. $\endgroup$ – shadowtalker Apr 11 '18 at 15:54
  • 2
    $\begingroup$ This opinionated answer is misleading in that random forest is a great option, especially since an RF can easily be class weighted. Please follow the helpful best practice of providing a counter proposal when saying no, otherwise saying no is more harmful than it is helpful. $\endgroup$ – SwimBikeRun Feb 9 '20 at 1:37

Unbalanced classes are only a an issue if you also have misclassification cost imbalance. If there are small minority classes and it is not more expensive to classify them as a majority class than the other way around, then the rational thing to do is to allow misclassification of minority classes.

So let's assume you have class and cost imbalance. There are multiple ways to deal with this. Max Kuhn's book "Applied predictive modeling" has a good overview in chapter 16. Those remedies include using a cutoff other than 0.5 which reflects the unequal costs. This is easy to do in binary classification as long as your classifier outputs label probabilities (trees and forests do this). I haven't looked into it for multiple classes yet. You can also oversample the minority class to give it more weight.

  • $\begingroup$ I don't think this is correct. If I have equal misclassification cost but my model is biased to overpredict one class, I am still left with a biased model at the end of the day. $\endgroup$ – shadowtalker Oct 29 '16 at 13:09
  • $\begingroup$ It wouldn't matter though. Cancer cases are much fewer than healthy patients. Yet you need to reliably predict the cancer patients because missing one is much more expensive than predicting one too many. If you had a data-set with 99.9% healthy people and 0.1% common cold cases, the best classifier would simply ignore those common cold cases. $\endgroup$ – David Ernst Oct 29 '16 at 13:57

Not the answer you're looking for? Browse other questions tagged or ask your own question.