0
$\begingroup$

Question: Why does the random forest do really well for the first class, then switch after 52 trees and do well for the second class, then switch again and do really well for the third class?

Background: I have a three-class classification problem with balanced classes (96000 samples from each class). I take 80% of the samples to use for training and 20% for testing. My class label is in the last column of the data frame.

Evidence of problem:

My call to R's randomForest package is:

model_1 <- randomForest(train_set[,-ncol(train_set)], train_set[,ncol(train_set)], test_set[,-ncol(test_set)], test_set[,ncol(test_set)], ntree=500, do.trace=2)

 ntree      OOB      1      2      3 |    Test      1      2      3
 2:       61.22% 47.10% 91.36% 45.28%|  60.98% 46.88% 91.12% 44.93%
 4:       61.14% 47.39% 91.13% 44.95%|  60.79% 46.70% 90.42% 45.24%
 6:       61.33% 27.38% 90.84% 65.79%|  60.78%  2.13% 90.27% 89.95%
 8:       61.28% 24.58% 85.06% 74.21%|  60.70%  1.93% 90.28% 89.89%
10:       61.17% 22.91% 80.52% 80.07%|  60.62%  1.71% 90.28% 89.87%
...
52:       61.17% 50.82% 57.57% 75.12%|  60.59%  1.60% 90.27% 89.90%
54:       61.14% 57.27% 48.15% 78.00%|  60.66% 91.60%  0.46% 89.90%
56:       61.15% 63.71% 39.00% 80.72%|  60.66% 91.64%  0.44% 89.90%
...
500:      61.77% 85.92% 58.23% 41.15%|  60.63% 91.61% 90.20%  0.07%
$\endgroup$

1 Answer 1

0
$\begingroup$

So, turns out I had fewer instances of each class than I thought. I checked it over and found an extra 0 (I was modelling as if I had 96,000 instances of each class instead of 9,600). If anyone else runs into this kind of flipping problem, check your number of samples per class!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.