I'm trying to fit a Random Forest model in R. I've got a DF with 13 factorized variables plus the target one (binary). When I try to use all the variables with
fitRf <- randomForest(fl_ok ~ ., data=train)
The result printing the model is
OOB estimate of error rate: 35.35%
Confusion matrix:
0 1 class.error
0 6492 263 0.03893412
1 4546 2303 0.66374653
The table with variables importance is
PROVINCIA_RESIDENZA_CONTRAENTE 989.2589
REGIONE 550.9323
FASCIA_ETA 602.3096
SESSO 153.0396
COD_CATEGORIA_VEICOLO 150.4787
COD_ZONA_DANNEGGIATA 562.0613
RUOLO 293.8663
TIPO_SX 99.2244
NUM_FIRME 132.7890
TIPO_LESIONE 506.2868
int_legale 673.7938
gg_ap_den_lesioni 260.0141
life_cycle 445.2724
When I try to leave out the most important variable ('PROVINCIA_RESIDENZA_CONTRAENTE') the model performance is better:
OOB estimate of error rate: 29.37%
Confusion matrix:
0 1 class.error
0 4594 2161 0.3199112
1 1835 5014 0.2679223
Note that the left out variable is a factor with 45 levels representing the County of the single record. Why is this happening? Is there anything I can do to use this variable in the model?