Skip to main content
Make point less garbled
Source Link
Dex Groves
  • 1.7k
  • 12
  • 12

RF handles factors by one-hot encoding them. It makes one new dummy column for every level of the factor variable. When there are new or different factor levels in a scoring dataframe, bad things happen.

If the train and test existed together in the same data structure at the point that the factor was defined, there isn't a problem. When the test has its factor defined separately then you get issues.

library("randomForest")

# Fit an RF on a few numerics and a factor. Give test set a new level.
N <- 100
df <- data.frame(num1 = rnorm(N), 
                 num2 = rnorm(N), 
                 fac = sample(letters[1:4], N, TRUE),
                 y = rnorm(N),
                 stringsAsFactors = FALSE)
df[100, "fac"] <- "a suffusion of yellow"
df$fac <- as.factor(df$fac)

train <- df[1:50, ]
test <- df[51:100, ]

rf <- randomForest(y ~ ., 
                   data=train)

# This is fine, even though the "yellow" level doesn't exist in train, RF
# is aware that it is a valid factor level
predict(rf, test)

# This is not fine. The factor level is introduced _after_and modelRF trainingcan't know of it
test$fac <- as.character(test$fac)
test[50, "fac"] <- "toyota corolla"
test$fac <- as.factor(test$fac)
predict(rf, test)

You can get around this issue by relevelling your scoring factors to match the training data.

# Can get around by relevelling the new factor. "toyota corolla" becomes NA
test$fac <- factor(test$fac, levels = levels(train$fac))
predict(rf, test)

RF handles factors by one-hot encoding them. It makes one new dummy column for every level of the factor variable. When there are new or different factor levels in a scoring dataframe, bad things happen.

If the train and test existed together in the same data structure at the point that the factor was defined, there isn't a problem. When the test has its factor defined separately then you get issues.

library("randomForest")

# Fit an RF on a few numerics and a factor. Give test set a new level.
N <- 100
df <- data.frame(num1 = rnorm(N), 
                 num2 = rnorm(N), 
                 fac = sample(letters[1:4], N, TRUE),
                 y = rnorm(N),
                 stringsAsFactors = FALSE)
df[100, "fac"] <- "a suffusion of yellow"
df$fac <- as.factor(df$fac)

train <- df[1:50, ]
test <- df[51:100, ]

rf <- randomForest(y ~ ., 
                   data=train)

# This is fine, even though the "yellow" level doesn't exist in train, RF
# is aware that it is a valid factor level
predict(rf, test)

# This is not fine. The factor level is introduced _after_ model training
test$fac <- as.character(test$fac)
test[50, "fac"] <- "toyota corolla"
test$fac <- as.factor(test$fac)
predict(rf, test)

You can get around this issue by relevelling your scoring factors to match the training data.

# Can get around by relevelling the new factor. "toyota corolla" becomes NA
test$fac <- factor(test$fac, levels = levels(train$fac))
predict(rf, test)

RF handles factors by one-hot encoding them. It makes one new dummy column for every level of the factor variable. When there are new or different factor levels in a scoring dataframe, bad things happen.

If the train and test existed together in the same data structure at the point that the factor was defined, there isn't a problem. When the test has its factor defined separately then you get issues.

library("randomForest")

# Fit an RF on a few numerics and a factor. Give test set a new level.
N <- 100
df <- data.frame(num1 = rnorm(N), 
                 num2 = rnorm(N), 
                 fac = sample(letters[1:4], N, TRUE),
                 y = rnorm(N),
                 stringsAsFactors = FALSE)
df[100, "fac"] <- "a suffusion of yellow"
df$fac <- as.factor(df$fac)

train <- df[1:50, ]
test <- df[51:100, ]

rf <- randomForest(y ~ ., data=train)

# This is fine, even though the "yellow" level doesn't exist in train, RF
# is aware that it is a valid factor level
predict(rf, test)

# This is not fine. The factor level is introduced and RF can't know of it
test$fac <- as.character(test$fac)
test[50, "fac"] <- "toyota corolla"
test$fac <- as.factor(test$fac)
predict(rf, test)

You can get around this issue by relevelling your scoring factors to match the training data.

# Can get around by relevelling the new factor. "toyota corolla" becomes NA
test$fac <- factor(test$fac, levels = levels(train$fac))
predict(rf, test)
Source Link
Dex Groves
  • 1.7k
  • 12
  • 12

RF handles factors by one-hot encoding them. It makes one new dummy column for every level of the factor variable. When there are new or different factor levels in a scoring dataframe, bad things happen.

If the train and test existed together in the same data structure at the point that the factor was defined, there isn't a problem. When the test has its factor defined separately then you get issues.

library("randomForest")

# Fit an RF on a few numerics and a factor. Give test set a new level.
N <- 100
df <- data.frame(num1 = rnorm(N), 
                 num2 = rnorm(N), 
                 fac = sample(letters[1:4], N, TRUE),
                 y = rnorm(N),
                 stringsAsFactors = FALSE)
df[100, "fac"] <- "a suffusion of yellow"
df$fac <- as.factor(df$fac)

train <- df[1:50, ]
test <- df[51:100, ]

rf <- randomForest(y ~ ., 
                   data=train)

# This is fine, even though the "yellow" level doesn't exist in train, RF
# is aware that it is a valid factor level
predict(rf, test)

# This is not fine. The factor level is introduced _after_ model training
test$fac <- as.character(test$fac)
test[50, "fac"] <- "toyota corolla"
test$fac <- as.factor(test$fac)
predict(rf, test)

You can get around this issue by relevelling your scoring factors to match the training data.

# Can get around by relevelling the new factor. "toyota corolla" becomes NA
test$fac <- factor(test$fac, levels = levels(train$fac))
predict(rf, test)