In Multinomial Naive Bayes Classifier, which parameter estimation do we use, is it Maximum Likelihood or Maximum A Posteriori?

If any one of the esteemed members may kindly help me out.


For learning the NBC, the ML estimate for feature $F_i$ given class $C_j$ is often used. That is $$ P(F_i \mid C_j) \leftarrow \frac{\text{# cases from class $C_j$ with feature $F_i$}}{\text{# cases from class $C_j$}}. $$ There are usually two options for setting the class marginals.. either $$ P(C_j) \leftarrow \frac{1}{\text{# of possible classes}}, $$ or $$ P(C_j) \leftarrow \frac{\text{# cases from class $C_j$}}{\text{# of cases}}. $$ The latter is the ML estimate for the marginal, and the former is just a nameless 'objective' approach. In this setup, there is no MAP estimate unless a prior is incorporated, but that is nonstandard.

  • $\begingroup$ Thank you. I was of this view. NLTK is implementing ELEProbdist sort of MLE, but Manning & Schutze recommended for MAP, though I did not find any reason for it. If additive smoothing has to be introduced then why MAP, it can be done with MLE also. Isn't it? Regards, Subhabrata Banerjee. $\endgroup$ – HIGGINS Jun 25 '14 at 22:03
  • $\begingroup$ MAP in obtained by using a prior. In this case when the prior is uniform, this is equivalent with additive smoothing, or Laplace smoothing. So, saying that you do not need MAP because you use MLE with smoothing is wrong, because MLE with smoothing is a particular case of MAP. Also, MAP can be used also to incorporate other kind of priors. Also, you can use a mean estimator, which is a combination of prior mean and MLE. And, finally, both of them are not fully Bayesian since are point estimates. A full treatment would involve to integrate out the parameters. $\endgroup$ – rapaio Jul 1 '14 at 9:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.