Let's say I have a variable that perfectly predicts one of the classes in my dataset:
set.seed(668130) dat <- iris dat$X <- sample(1:3, nrow(iris), replace=TRUE) dat$X <- ifelse(dat$Species=='setosa', 1, dat$X) > table(dat$X, dat$Species) setosa versicolor virginica 1 50 12 15 2 0 18 15 3 0 20 20
Why does the NaiveBayes algorithm fail on this dataset?
library(klaR) > NaiveBayes(Species ~ ., dat) Error in NaiveBayes.default(X, Y, ...) : Zero variances for at least one class in variables: X
It seems to me that it would be reasonable to output a classification of 'setosa' 100% of the time, if X=1. Other algorithms (such as randomForest) do this:
library(randomForest) > randomForest(Species ~ ., dat) Call: randomForest(formula = Species ~ ., data = dat) Type of random forest: classification Number of trees: 500 No. of variables tried at each split: 2 OOB estimate of error rate: 4.67% Confusion matrix: setosa versicolor virginica class.error setosa 50 0 0 0.00 versicolor 0 47 3 0.06 virginica 0 4 46 0.08
Is the NaiveBayes algorithm mathematically undefined in this case? I know the specific dataset is a little contrived, but the problem pops up occasionally when I am cross-validating NaiveBayes models.