In Multinomial Naive Bayes Classifier, which parameter estimation do we use, is it Maximum Likelihood or Maximum A Posteriori?
If any one of the esteemed members may kindly help me out.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
For learning the NBC, the ML estimate for feature $F_i$ given class $C_j$ is often used. That is $$ P(F_i \mid C_j) \leftarrow \frac{\text{# cases from class $C_j$ with feature $F_i$}}{\text{# cases from class $C_j$}}. $$ There are usually two options for setting the class marginals.. either $$ P(C_j) \leftarrow \frac{1}{\text{# of possible classes}}, $$ or $$ P(C_j) \leftarrow \frac{\text{# cases from class $C_j$}}{\text{# of cases}}. $$ The latter is the ML estimate for the marginal, and the former is just a nameless 'objective' approach. In this setup, there is no MAP estimate unless a prior is incorporated, but that is nonstandard.
-
$\begingroup$ Thank you. I was of this view. NLTK is implementing ELEProbdist sort of MLE, but Manning & Schutze recommended for MAP, though I did not find any reason for it. If additive smoothing has to be introduced then why MAP, it can be done with MLE also. Isn't it? Regards, Subhabrata Banerjee. $\endgroup$– HIGGINSCommented Jun 25, 2014 at 22:03
-
$\begingroup$ MAP in obtained by using a prior. In this case when the prior is uniform, this is equivalent with additive smoothing, or Laplace smoothing. So, saying that you do not need MAP because you use MLE with smoothing is wrong, because MLE with smoothing is a particular case of MAP. Also, MAP can be used also to incorporate other kind of priors. Also, you can use a mean estimator, which is a combination of prior mean and MLE. And, finally, both of them are not fully Bayesian since are point estimates. A full treatment would involve to integrate out the parameters. $\endgroup$– rapaioCommented Jul 1, 2014 at 9:11