I'm using random forest and the out of bag error for the level of one class is very different to its test error. I'm working with a cutt-of equal to c(0.2,0.8). Here's the case:
fmla <- as.formula(paste("ex ~ ", paste(names(muestra.fullarbol[,-c(1,2,3,9,10,11,12,17,19,20,21,22,23,24,29,31,32,33,34,35,36,47)]), collapse= "+")))
> bosque <- randomForest(fmla , data=muestra.fullarbol ,mtry=12, ntree=1000 , cutoff=c(0.2,0.8),importance=TRUE)
> bosque
Call:
randomForest(formula = fmla, data = muestra.fullarbol, mtry = 12, ntree = 1000, cutoff = c(0.2, 0.8), importance = TRUE)
Type of random forest: classification
Number of trees: 1000
No. of variables tried at each split: 12
OOB estimate of error rate: 15.81%
Confusion matrix:
No Si class.error
No 3999 1 0.00025
Si 758 42 0.94750
As we see, the out of bag error for the level "SI" is 0.94750. However If I use the test set to get a sense of the error, the result is very different:
res.arbol <- predict(bosque,test.fullarbol,type="class")
> summary(res.arbol)
No Si
7761 43
> table(res.arbol,test.fullarbol$ex)
res.arbol No Si
No 6937 824
Si 4 39
> prop.table(table(res.arbol,test.fullarbol$ex),1)
res.arbol No Si
No 0.89382811 0.10617189
Si 0.09302326 0.90697674
Now we see the error rate for the test set in the class "Si" is very low , it's equal to 0.093 and It doesn´t make sense to me.
I guess that the cut off is working just to predict out of the sample(muestra.fullarbol), but I'm not sure. What can be the reason of that huge difference?