Random forest: how to handle new factor levels in test set? I am trying to make predictions using a random forest model in R.
However I get errors since some factors have different values in the test set than in the training set. For example, a factor Cat_2 has values 34, 68, 76, etc., in the test set that do not appear in the training set. Unfortunately, I do not have control over the Test set... I must use it as-is.
My only workaround was to convert the problematic factors back to numerical values, using as.numeric(). It works but I am not very satisfied, since these values are codes that have no numerical sense...
Do you think there would be another solution, to drop the new values from the test set? But without removing all the other factor values (let say values 1, 2, 14, 32, etc.) which are in both training and test, and contains information potentially useful for predictions.
 A: King and Bonoit, this snippet can be useful to harmonize levels:
for(attr in colnames(training))
{
  if (is.factor(training[[attr]]))
  {
    new.levels <- setdiff(levels(training[[attr]]), levels(testing[[attr]]))
    if ( length(new.levels) == 0 )
    { print(paste(attr, '- no new levels')) }
    else
    {
      print(c(paste(attr, length(new.levels), 'of new levels, e.g.'), head(new.levels, 2)))
      levels(testing[[attr]]) <- union(levels(testing[[attr]]), levels(training[[attr]]))
    }
  }
}

It also prints which attributes are changed.
I did not find a good way to write it more elegantly (with ldply or something). Any tips are appreciated.
A: Here's some code I wrote that addresses @King's response above. It fixed the error:
# loops through factors and standardizes the levels
for (f in 1:length(names(trainingDataSet))) {
    if (levels(testDataSet[,f]) > levels(trainingDataSet[,f])) {    
            levels(testDataSet[,f]) = levels(trainingDataSet[,f])       
    } else {
            levels(trainingDataSetSMOTEpred[,f]) = levels(testDataSet[,f])      
    }
}

A: Test and training set should be combined as one set and then change the levels of the training set.  My codes are:
totalData <- rbind(trainData, testData)
for (f in 1:length(names(totalData))) {
  levels(trainData[, f]) <- levels(totalData[, f])
}

This works in any cases where the number of levels in test are more or less than training. 
A: If the test set has a lot of these points with new factor values then I'm not sure what the best approach is. If it is just a handful of points you might be able to get away with something fudgy like treating the errant factor levels as missing data and imputing them with whatever approach you see fit. The R implementation has a couple of ways to impute missing data, you just need to set these factor levels to NA to indicate they are missing.
A: I have a lousy workaround when I use randomForest in R. It's probably not theoretically sound, but it gets the thing running.
levels(testSet$Cat_2) = levels(trainingSet$Cat_2)

or the other way round. Basically, it just gives tells R that it's a valid value just that there are 0 cases; so stop bugging me about the error.
I'm not smart enough to code it such that it automatically performs the action for all categorical features. Send me the code if you know how...
A: If your test data has some levels that are an insignificant percentage of the population,  but are just messing up your ability to make the model, here is an elegant way to remove the extraneous levels from the test set:
uniquetrain <- unique( train$category)
test <- test[test$category %in% uniquetrain,]

A: I'm sure that you would have thought of this already if this were the case, but if the test set has actual values and you're using the test set for cross validation purposes, then re-splitting the dataframe into training and test datamframes where the two are balanced on these factors would avoid your problem. This method is popularly known as stratified cross-validation. 
