Naive Bayesian Algorithm in R/SAS for categorical input variables?

Question

Could anyone please let me know how to implement Naive Bayesian Algorithm in R or SAS?I have got a training dataset with all the categorical predictors and target variable(3 levels).I got to build a model and apply it on a different test dataset along with the probable target and its predicted probability.

To be more clear,my first dataset 'A' contact 4 categorical input variables a,b,c,d and a target class 'T' of 3 levels.I need to train the model for this dataset initially.Then,I have got one more dataset 'B' with input categorical variables w,x,y,z and I need to predict the probable target class 'S' along with its probability here based on my previous built model.I want the entire thing to be done in R or SAS but couldn't find much resources.Sorry,if the question has been repeated.

Answer 1

have you looked at e1071 package in R?

naiveBayes(...) and predict(...) functions do exactly what you're looking for.

Answer 2

I found "Machine Learning with R" by Brett Lantz (Packt Publishing) a good intro into how to code many representitive ML algos in R. There's a chapter on Naive Bayes.

http://www.packtpub.com/machine-learning-with-r/book

Using that book for a reference to solve your model,

1. Import the data into R

   (assuming it's in a CSV file. There are several ways data can be read into R)

   # set stringsAsFactors = TRUE as all your inputs are categorical
   A_raw <- read.csv("A.csv", stringsAsFactors = TRUE)  
   

   This will create a data frame object which you can see the properties using str(A_raw).

   You should have 5 columns, a,b,c,d and T and all should be of type Factor.

2. create trainging and test datasets

   Say you have 1000 rows in your data set, you can use 80% for training and 20% for testing. Before dividing the data first you need to make sure that the rows are in a random order

    set.seed(12345) # Randomize the data
    A_rand <- A_raw[order(runif(1000)), ]
   
    # split the data between training and test data
    A_train <- A_rand[1:900, ]
    A_test <- A_rand[901:1000, ]
   

   You can confirm you have a similar percentage of each type of T in both the training set and testing set

   prop.table(table(A_train$T))
   prop.table(table(A_test$T))
   

3. Use the Naive Bayes function from the e1071 package

   A_classifier <- naiveBayes(A_train, A_train$T)
   

4. Evaluate the model:

   A_predicted <- predict(A_classifier, A_test)
   

   You can then use CrossTable from the gmodels library to compare the predicited values to the actual values

   CrossTable(A_predicted, A_test$T, prop.chisq = FALSE, prop.t = FALSE, dnn = c('predicted', 'actual'))
   

5. Improve the model performance

   One way (which the book mentions) is to set a value for the Laplace estimator when training the model but this really only is applicable when there is a large number of input variables and there is a chance that there could be zero instances of 1 of the inputs associated with one of the outcomes. In your case when there is only 4, then it is probably unlikely. In case you wanted to set the Laplace variable you can do this:

   A_classifier2 <- naiveBayes(A_train, A_train$T, laplace = 1)
   

Of course you can then apply this method for dataset B too.

Answer 3

@Paul P M and @omidi already mention the package you can use for a ready-to-use implementation of Naive Bayes.

But if I am not mistaken, you are asking for instructions on how to implement the algorithm yourself. In that case, you can have a look at this brilliant answer by @Matt Krause. Assuming you can code in R, implementing the algorithm should not be very hard after reading Matt's explanations on the link given.