0
$\begingroup$

I have trained my data with naiveBayes algo in e1071 package. I have 6 classes in my data. I have predicted test data. the prediction returns only one class for each data point but I would like to know the probability of all 6 classes for each data point in test set. Please help me to know how to calculate the same.

$\endgroup$
0
3
$\begingroup$

As mentioned in the documentation, you have to set type='raw' within predict.

-type:

If "raw", the conditional a-posterior probabilities for each class are returned, and the class with maximal probability else.

About the needs for Laplace smoothing (from Wikipedia Naive Bayes paper):

If a given class and feature value never occur together in the training data, then the frequency-based probability estimate will be zero. This is problematic because it will wipe out all information in the other probabilities when they are multiplied. Therefore, it is often desirable to incorporate a small-sample correction, called pseudocount, in all probability estimates such that no probability is ever set to be exactly zero. This way of regularizing naive Bayes is called Laplace smoothing.

From the Wikipedia Laplace smoothing paper:

The pseudocount $\alpha > 0$ (laplace parameter in R function) is the smoothing parameter ($\alpha = 0$ corresponds to no smoothing). Some authors have argued that $\alpha$ should be 1, though in practice a smaller value is typically chosen.

From the examples in the R documentation, we see that laplace can also be set to higher values such as 3.

$\endgroup$
3
  • $\begingroup$ After setting type="raw" I get: class 1: NaN class 2: NaN class 3: NaN class 4: NaN class 5: NaN class 6: NaN $\endgroup$ Jul 21 '16 at 9:20
  • $\begingroup$ After laplace transform it's working. Can you please tell me what should be the appropriate value for parameter laplace = "?" $\endgroup$ Jul 21 '16 at 9:42
  • $\begingroup$ basically when you have unknown words in your test, set without Laplace smoothing, you get zero probabilities (see for instance stats.stackexchange.com/questions/108797/…). I have also edited my answer. From the documentation examples it seems that laplace=3 works $\endgroup$
    – Antoine
    Jul 21 '16 at 10:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.