0
$\begingroup$

I'm using Random Forest for classification which gives the following confusion matrix.

0 1 class.error

0 839 24 0.027

1 60 86 0.410

You can notice that the classification error is a lot higher for incorrect classification of 1's (false negatives).

I've noticed - even when applying RF to other problems - that the classification error tends be higher on the basis of the proportions of the dependent variable. For eg:- in the example above, there are 146 cases with DV = 1 as opposed to 863 cases with DV = 0 so the error for classifying a case as 1 is much higher

My question is this: What is the reason that the RF algorithm behaves this way and how can I improve the results to remove what seems to me like a bias.

$\endgroup$

1 Answer 1

4
$\begingroup$

(1) almost all classification algorithms assume an equal cost of misclassification. Will thus be dependent of the ratio of 1/0 in the dependent variable. Will thus often not be useful (without adjustment) when the ratio of outcomes is large.
(2) this is known known issue. use cost sensitive models (C50) or oversample minority case /oversample minority case to impose a different (your preferred) cost ratio onto the model, rather than using default 1:1 ratio

Update:
(1) the Kunh book: Applied Predictive Modeling - has a chapter on this
(2) for random forests, the Andy Liaw paper is often referenced: Using Random Forest to Learn Imbalanced Data

$\endgroup$
3
  • $\begingroup$ Thanks a lot!. Would you happen to have links some documentation regarding this? $\endgroup$ Commented Aug 25, 2016 at 14:51
  • $\begingroup$ updated answer - hope useful $\endgroup$
    – charles
    Commented Aug 26, 2016 at 7:41
  • 3
    $\begingroup$ Classification is seldom the approach that answers the underlying question, and doesn't work for rare outcomes. See fharrell.com/post/classification. $\endgroup$ Commented Sep 30, 2021 at 11:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.