Skip to main content
7 of 7
added 306 characters in body; edited title

Prediction using Naive Bayes of klaR package fails

I am trying to replicate a example that I found in Tom Mitchell's book Machine Learning (1997), using R. It is a example from chapter 6.

There are 14 training examples (shown below) of the target concept PlayTennis, where each day is described by the attributes Outlook, Temperature, Humidity, and Windy.

Training examples:

Outlook,Temperature,Humidity,Windy,Play
overcast,cool,normal,true,yes
overcast,hot,high,false,yes
overcast,hot,normal,false,yes
overcast,mild,high,true,yes
rainy,cool,normal,false,yes
rainy,mild,high,false,yes
rainy,mild,normal,false,yes
sunny,cool,normal,false,yes
sunny,mild,normal,true,yes
rainy,cool,normal,true,no
rainy,mild,high,true,no
sunny,hot,high,false,no
sunny,hot,high,true,no
sunny,mild,high,false,no

Here's my code:

library("klaR")
library("caret")

data = read.csv("example.csv")

x = data[,-5]
y = data$Play

model = train(x,y,'nb',trControl=trainControl(method='cv',number=10))

Outlook <- "sunny"
Temperature <- "cool"
Humidity <- "high"
Windy <- "true"

instance <- data.frame(Outlook,Temperature,Humidity,Windy)

predict(model$finalModel,instance)

The example tries to predict the outcome for

Outlook=sunny, Temperature=cool,Humidity=high and Wind=strong

The problem is that I am getting a different prediction from the one in the book.

Here are the probabilities I've got from my code:

no          yes
0.001078835 0.9989212

Here are the book's probabilities:

no     yes
0.0206 0.0053

My code classifies the unseen data as Yes and the book's classifier classifies it as No.

Shouldn't both give the same answer since we are using the same naive Bayes classifier?

EDIT:

I replicated the example using scikit-learn MultinomialNB classifier and I have got the following probabilities

no    yes
0.769  0.231

which are similar to the normalized probabilities of the book.

Normalized probabilities of the book

no     yes
0.795  0.205