RF handles factors by one-hot encoding them. It makes one new dummy column for every level of the factor variable. When there are new or different factor levels in a scoring dataframe, bad things happen.
If the train and test existed together in the same data structure at the point that the factor was defined, there isn't a problem. When the test has its factor defined separately then you get issues.
library("randomForest")
# Fit an RF on a few numerics and a factor. Give test set a new level.
N <- 100
df <- data.frame(num1 = rnorm(N),
num2 = rnorm(N),
fac = sample(letters[1:4], N, TRUE),
y = rnorm(N),
stringsAsFactors = FALSE)
df[100, "fac"] <- "a suffusion of yellow"
df$fac <- as.factor(df$fac)
train <- df[1:50, ]
test <- df[51:100, ]
rf <- randomForest(y ~ .,
data=train)
# This is fine, even though the "yellow" level doesn't exist in train, RF
# is aware that it is a valid factor level
predict(rf, test)
# This is not fine. The factor level is introduced _after_and modelRF trainingcan't know of it
test$fac <- as.character(test$fac)
test[50, "fac"] <- "toyota corolla"
test$fac <- as.factor(test$fac)
predict(rf, test)
You can get around this issue by relevelling your scoring factors to match the training data.
# Can get around by relevelling the new factor. "toyota corolla" becomes NA
test$fac <- factor(test$fac, levels = levels(train$fac))
predict(rf, test)