# Naive Bayesian Algorithm in R/SAS for categorical input variables?

Could anyone please let me know how to implement Naive Bayesian Algorithm in R or SAS?I have got a training dataset with all the categorical predictors and target variable(3 levels).I got to build a model and apply it on a different test dataset along with the probable target and its predicted probability.

To be more clear,my first dataset 'A' contact 4 categorical input variables a,b,c,d and a target class 'T' of 3 levels.I need to train the model for this dataset initially.Then,I have got one more dataset 'B' with input categorical variables w,x,y,z and I need to predict the probable target class 'S' along with its probability here based on my previous built model.I want the entire thing to be done in R or SAS but couldn't find much resources.Sorry,if the question has been repeated.

have you looked at e1071 package in R?

naiveBayes(...) and predict(...) functions do exactly what you're looking for.

I found "Machine Learning with R" by Brett Lantz (Packt Publishing) a good intro into how to code many representitive ML algos in R. There's a chapter on Naive Bayes.

http://www.packtpub.com/machine-learning-with-r/book

Using that book for a reference to solve your model,

1. Import the data into R

(assuming it's in a CSV file. There are several ways data can be read into R)

# set stringsAsFactors = TRUE as all your inputs are categorical
A_raw <- read.csv("A.csv", stringsAsFactors = TRUE)


This will create a data frame object which you can see the properties using str(A_raw).

You should have 5 columns, a,b,c,d and T and all should be of type Factor.

2. create trainging and test datasets

Say you have 1000 rows in your data set, you can use 80% for training and 20% for testing. Before dividing the data first you need to make sure that the rows are in a random order

 set.seed(12345) # Randomize the data
A_rand <- A_raw[order(runif(1000)), ]

# split the data between training and test data
A_train <- A_rand[1:900, ]
A_test <- A_rand[901:1000, ]


You can confirm you have a similar percentage of each type of T in both the training set and testing set

prop.table(table(A_train$T)) prop.table(table(A_test$T))

3. Use the Naive Bayes function from the e1071 package

A_classifier <- naiveBayes(A_train, A_train$T)  4. Evaluate the model: A_predicted <- predict(A_classifier, A_test)  You can then use CrossTable from the gmodels library to compare the predicited values to the actual values CrossTable(A_predicted, A_test$T, prop.chisq = FALSE, prop.t = FALSE, dnn = c('predicted', 'actual'))

5. Improve the model performance

One way (which the book mentions) is to set a value for the Laplace estimator when training the model but this really only is applicable when there is a large number of input variables and there is a chance that there could be zero instances of 1 of the inputs associated with one of the outcomes. In your case when there is only 4, then it is probably unlikely. In case you wanted to set the Laplace variable you can do this:

A_classifier2 <- naiveBayes(A_train, A_train\$T, laplace = 1)


Of course you can then apply this method for dataset B too.

• Thanks a lot Paul for the inputs.Will definitely work on it and get back to you. – maddy Feb 13 '14 at 7:33

@Paul P M and @omidi already mention the package you can use for a ready-to-use implementation of Naive Bayes.

But if I am not mistaken, you are asking for instructions on how to implement the algorithm yourself. In that case, you can have a look at this brilliant answer by @Matt Krause. Assuming you can code in R, implementing the algorithm should not be very hard after reading Matt's explanations on the link given.