Bayes' theorem [is used] in the classifier's decision rule, but naive Bayes is not (necessarily) a Bayesian method.


I have applied naive Bayes assumption to LDA, but it has a Bayes classifier as the sole classifier. To what other non-Bayesian classifiers or learning models could the naive Bayes assumption be applied?


That line from Wikipedia gives two citations, the latter of which is to

Hand, D. J.; Yu, K. (2001). "Idiot's Bayes — not so stupid after all?". International Statistical Review. 69 (3): 385–399. doi:10.2307/1403452. ISSN 0306-7734.

An excerpt from the end of their section 1:

Before commencing, however, some clarification of the word 'Bayes' in this context is appropriate. In work on supervised classification methods it has become standard to refer to the $P(i)$ as the class $i$ prior probability, because this gives the probability that an object will belong to class $i$ prior to observing any information about the object. Combining the prior with $P(\mathbf x \mid i)$, as shown above, gives the posterior probability, after having observed $\mathbf x$. This combination is effected via Bayes theorem, and it is because of this that 'Bayes' is used in the name of these methods. [...] The important point for us is that no notion of subjective probability is introduced: the methods are not [necessarily] 'Bayesian' in the formal statistical sense. [...] In this paper, following almost all the work on the idiot's Bayes method, we adopt a frequentist interpretation.

The Wikipedia claim, I think, is just that naive Bayes need not be interpreted as a Bayesian method based on a subjective prior and so on; it makes perfect sense to think of it as a frequentist method. Similarly, note that the Bayes classifier is not an inherently Bayesian concept, but in fact comes up in frequentist analyses quite often; the naive Bayes classifier is simply the Bayes classifier under a certain independence assumption, and so that need not be a Bayesian method either.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.