New factors levels not present in training data

Question

I am getting the "New factors levels not present in training data" error. But I checked the nlevels and class for every column in development as well as test data and they are the same. Any plausible explanation?

Answer 1

RF handles factors by one-hot encoding them. It makes one new dummy column for every level of the factor variable. When there are new or different factor levels in a scoring dataframe, bad things happen.

If the train and test existed together in the same data structure at the point that the factor was defined, there isn't a problem. When the test has its factor defined separately then you get issues.

library("randomForest")

# Fit an RF on a few numerics and a factor. Give test set a new level.
N <- 100
df <- data.frame(num1 = rnorm(N), 
                 num2 = rnorm(N), 
                 fac = sample(letters[1:4], N, TRUE),
                 y = rnorm(N),
                 stringsAsFactors = FALSE)
df[100, "fac"] <- "a suffusion of yellow"
df$fac <- as.factor(df$fac)

train <- df[1:50, ]
test <- df[51:100, ]

rf <- randomForest(y ~ ., data=train)

# This is fine, even though the "yellow" level doesn't exist in train, RF
# is aware that it is a valid factor level
predict(rf, test)

# This is not fine. The factor level is introduced and RF can't know of it
test$fac <- as.character(test$fac)
test[50, "fac"] <- "toyota corolla"
test$fac <- as.factor(test$fac)
predict(rf, test)

You can get around this issue by relevelling your scoring factors to match the training data.

# Can get around by relevelling the new factor. "toyota corolla" becomes NA
test$fac <- factor(test$fac, levels = levels(train$fac))
predict(rf, test)

Answer 2

I just encountered the issue as well when using expand.grid() to examine predictions of randomForest() across various factor levels.

The issue is created by expand.grid() setting stringsAsFactors = T by default, which coerces strings to factors using the available levels of the data. This creates a problem when one is only using a subset of factor levels for predictions.

I fixed the issue by setting stringsAsFactors = F which then allows randomForest() to do the one hot encoding as the previous answer suggested.