2
$\begingroup$

Hello Crossvalidated!

I have a question that I can't figure out.

I am working on building a classifier for a dichotomous outcome (0, 1). I use R for this. I used a Random Forest algorithm from the randomForest package.

The outcome variable is very skewed with the "1" outcome only being 7% of the cases, with n = 12000. The results of the analysis resulted in a misclassification rate for the "1" of 97% and for the "0" below 0.1%

Since random forest is essentially a from of bootstrapping, I tried stratifying on the outcome to get better classifications. The misclassification rates changed to "1" 60% and "0" 20%.

My gut feeling tells me that stratifying on outcome is not good practice, but I can't seem to find anything specifically on the subject.

The answer below relates to this, but the situation is different. Most interesting statistical paradoxes

In the answer above a "reverse" Simpsons paradox is explained, but this is when one unknowingly stratifies on outcome.

So the question is: Are there any negative effects from stratifying on outcome when using a bootstrap/random forest?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.