Naive Bayes model [duplicate]

This question already has an answer here:

i am trying to implement the naive bayes algorithm in java. But i have a few confusions on the naive bayes model. What is the model of naive bayes? Is it a table ?how we can make prediction from the model? I can implement the algorithm in java code and make prediction.But i heard that's not the right way to do it. we should make prediction from the model ..but how ??If anyone have some idea about this,kindly please try to give an answer to this . And also, how can we split the training set and test set from the dataset ??? Thank you.

marked as duplicate by steffen, gung♦, Momo, COOLSerdash, csgillespieFeb 24 '14 at 17:48

• (-1) Did you look up Naive Bayes on the web before ? Do you understand how classification works ? What do you mean by ""heard that's not the right way to do it" ? Why ? What do you mean by "how can we split ..." ? – steffen Feb 24 '14 at 15:24

The question is slightly unclear, but I'll try to explain some about the concept and implementation of Naive Bayes.

I implemented a Naive Bayes Classifier in Java recently. The only real data set I have tested it against is the Titanic data set from kaggle.com, but it was able to get up to about 80% correct, depending on which dimensions I chose.

Basically the way I understand it (and the way I implemented is) you are trying to classify data into several different classifications, according to a series of characteristics (or dimensions or whatever you want to call them). The idea is that each classification has a certain probability distribution for each of the dimensions, and if you know the values of each dimension for a certain observation, you can work backwards to find the most likely classification using Bayes Rule. The reason it is called "naive" is because it assumes that each of the dimensions is independent. If this is not true, it can cause the results to be inaccurate.

From Bayes Rule we have $$P({\rm Classification} | {\rm Evidence}) = \frac{P({\rm Evidence} | {\rm Classification}) P({\rm Classification})}{P({\rm Evidence})}$$

where

$$P({\rm Evidence}) = \sum{P(E | {\rm classification}_i)P({\rm classification}_i)}$$.

If you have access to a training data set that tells which classification each observation belongs to, then you can find out P(Classification) simply by computing the frequencies of the various classifications, P(Evidence | Classification) by computing the frequencies of a set of evidence for each classification, and P(Evidence) by using the formula above. So it is just a matter of doing the math to find out P(Classification | Evidence).

As an example, look at the Titanic competition on Kaggle. The train.csv file has a series of columns, like sex, age, ticket class, which port they got on at, etc., as well as whether or not that particular person survived. So it is easy to find out P(survived) and P(did not survive) (there are only two classifications here). That is you P(Classification).

Let's say you are basing your analysis on sex, class, and port (called "embarked" in the csv). You can find out P(sex = male) and P(sex = female), P(class = 1), P(embarked = Cherbourg), etc simply by adding up the number of instances of each and dividing by the total.

Then, you must find the probability for each dimension for each classification - that is, P(sex = male | survived = 1), P(sex = male | survived = 0) and so forth. Finally, you can get P(Evidence | Classification) by doing something like P(sex = male, pclass = 3, embarked = C | survived = 0) == P(sex = male | survived = 0) * P(pclass = 3 | survived = 0) * P(embarked = C | survived = 0).

For P(Evidence), you could use the summation formula above, but in this case with the data set right in front of you, you can also multiply individual probabilities, since we are assuming they are independent. Then, the probability of a male in third class getting on at Cherbourg would be P(sex = male, pclass = 3, embarked = C) == P(sex = male) * P(pclass = 3) * P(embarked = C).

At this point you have P(Evidence | Classification), P(Classification), and P(Evidence), so it is just a matter of doing the math to find P(Classification | Evidence). You do this for each classification, and whichever has the highest probability is the one you go with.

Let me know if this does not answer your question.

You can read more about the algorithm here

The division of dataset into two partitions may be arbitrary. It would be proper to select %80 of the dataset as training, and %20 of the dataset as test data. (I assume you do not do any cross-validation which would require you to select a validation set.)