1
$\begingroup$

While studying the naive Bayes classifier, two questions came to mind:

  1. When (if this can happen) is the naive Bayes classifier equivalent to the Bayes classifier?
  2. Why do I usually read in textbooks that naive Bayes is popular? What makes it so "popular"?
$\endgroup$
1
  • $\begingroup$ The "naive" independence of features assumption makes the calculations multiples (or sums in log space) which is convenient to calculate. It is relatively easy to get class probabilities and provides a sort of baseline for model comparison. The popularity is because it would the model to "beat". $\endgroup$ – knk Dec 28 '18 at 22:08
2
$\begingroup$

I would try not to conflate naive Bayes and the concept of a Bayes classifier. The former is a specific kind of classification model, whereas the latter should really just be viewed as an "optimal" classifier in a given setting (it could be any type of model, so long as it's the "true" model).

The reason we call the optimal classifier a Bayes classifier is because the best classifier needs to use Bayesian updating when making predictions, by which we mean that we follow Bayes theorem (it is a theorem after all) when updating our expectations based on evidence.

To say that a naive Bayes classifier is the Bayes classifier would just mean that no classifier can perform better in terms of misclassification rate (that is, it "knows" all the marginal distributions of the predictors for each class and correctly assumes they're all independent).

To your second question, I believe naive Bayes gained popularity because it's easy to implement and historically (despite it's generally false assumptions) it often performed well at certain tasks, particularly text classification.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.