Naive Bayes fails with a perfect predictor

Question

Let's say I have a variable that perfectly predicts one of the classes in my dataset:

set.seed(668130)
dat <- iris
dat$X <- sample(1:3, nrow(iris), replace=TRUE)
    dat$X  <- ifelse(dat$Species=='setosa', 1, dat$X)
> table(dat$X, dat$Species)

    setosa versicolor virginica
  1     50         12        15
  2      0         18        15
  3      0         20        20

Why does the NaiveBayes algorithm fail on this dataset?

library(klaR)
> NaiveBayes(Species ~ ., dat)
Error in NaiveBayes.default(X, Y, ...) : 
  Zero variances for at least one class in variables: X

It seems to me that it would be reasonable to output a classification of 'setosa' 100% of the time, if X=1. Other algorithms (such as randomForest) do this:

library(randomForest)
> randomForest(Species ~ ., dat)

Call:
 randomForest(formula = Species ~ ., data = dat) 
               Type of random forest: classification
                     Number of trees: 500
No. of variables tried at each split: 2

        OOB estimate of  error rate: 4.67%
Confusion matrix:
           setosa versicolor virginica class.error
setosa         50          0         0        0.00
versicolor      0         47         3        0.06
virginica       0          4        46        0.08

Is the NaiveBayes algorithm mathematically undefined in this case? I know the specific dataset is a little contrived, but the problem pops up occasionally when I am cross-validating NaiveBayes models.

Answer 1

Note that dat$X in your code is a numeric variable. The NaiveBayes implementation in klaR for numeric predictor variables calculates the mean and standard deviations of the predictor variable at each outcome level. Rather than dealing with standard deviations of 0, the klaR authors decided to throw an error.

If you change dat$X to a factor, it will create classification tables you are likely expecting. Alternatively, the naiveBayes function in the e1071 package will return distributions with a standard deviation of 0 if you prefer that over throwing errors, or you can delete the stop(...) code towards the end of klaR:::NaiveBayes.default (though that might cause problems with prediction and plotting functions in klaR).