# Random forest and new factor levels in test set

I am trying to make predictions using a Random forest model in R.

However I get errors since some factors have different values in the test set than in the training set. E.g. a factor "Cat_2" have values 34,68,76,etc in the test set that do not appear in the training set

Unfortunately, I do not have control over the Test set... I must use it as-is.

My only workaround was to convert the problematic factors back to numerical values, using as.numeric() It works but I am not very satisfied, since these values are codes that have no numerical sense...

Do you think there would be another solution, to drop the new values from the test set? But without removing all the other factor values (let say values 1,2,14,32,etc) which are in both training and test, and contains information potentially useful for predictions.

-
I see know reason why values in the test would have to be in the training set. The idea of classification is to use the training data to get an idea of what the class-conditional densities look like. You don't get to see every possible value from the density. I a variable is used in a split ona tree then the split determines which branch to follow for any unseen values as well as those that have been seen. –  Michael Chernick May 30 '12 at 0:04
You make a valid point, but on a practical level using the specific tool enquired about (the RF package in R) this is not allowed. My answer involving imputation is one way around it, though certainly not the best solution. Is does at least make the code not crash, so at least works, for small values of work. –  Bogdanovist May 30 '12 at 2:13
Similar to my question here: stats.stackexchange.com/questions/18004/…. I think I might use GBM instead of RF as it seems to deal with new factor levels better. Also, have you looked at the implementation of RF in party? I have never liked randomForest because of these issues (and inability to seamlessly deal with missing values). –  B_Miner May 31 '12 at 1:13

If the test set has a lot of these points with new factor values then I'm not sure what the best approach is. If it is just a handful of points you might be able to get away with something fudgy like treating the errant factor levels as missing data and imputing them with whatever approach you see fit. The R implementation has a couple of ways to impute missing data, you just need to set these factor levels to NA to indicate they are missing.

-
levels(testSet$Cat_2) = levels(trainingSet$Cat_2)