0
$\begingroup$

I'm using Random Forest for classification which gives the following confusion matrix.

0 1 class.error

0 839 24 0.027

1 60 86 0.410

You can notice that the classification error is a lot higher for incorrect classification of 1's (false negatives).

I've noticed - even when applying RF to other problems - that the classification error tends be higher on the basis of the proportions of the dependent variable. For eg:- in the example above, there are 146 cases with DV = 1 as opposed to 863 cases with DV = 0 so the error for classifying a case as 1 is much higher

My question is this: What is the reason that the RF algorithm behaves this way and how can I improve the results to remove what seems to me like a bias.

$\endgroup$
0
$\begingroup$

(1) almost all classification algorithms assume an equal cost of misclassification. Will thus be dependent of the ratio of 1/0 in the dependent variable. Will thus often not be useful (without adjustment) when the ratio of outcomes is large.
(2) this is known known issue. use cost sensitive models (C50) or oversample minority case /oversample minority case to impose a different (your preferred) cost ratio onto the model, rather than using default 1:1 ratio

Update:
(1) the Kunh book: Applied Predictive Modeling - has a chapter on this
(2) for random forests, the Andy Liaw paper is often referenced: Using Random Forest to Learn Imbalanced Data

$\endgroup$
  • $\begingroup$ Thanks a lot!. Would you happen to have links some documentation regarding this? $\endgroup$ – Rahul Singh Aug 25 '16 at 14:51
  • $\begingroup$ updated answer - hope useful $\endgroup$ – charles Aug 26 '16 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.