I've looking around Google Scholar for the earliest mention of this particular classifier and have not had much luck finding a definitive source. I've seen some sources cite as late as the 1980s and other as early as the 1930s. Does anyone know when the Naïve Bayes Classifier was developed and/or first used as a classification technique?
2
-
4$\begingroup$ I am afraid that this is such a simple idea (yet not trivial) that it might have been rediscovered many times, most likely even without being explicitly highlighted -- thus you are rather looking for the first work which introduces the term "Naive Bayes". $\endgroup$– user88Commented Nov 11, 2011 at 16:23
-
1$\begingroup$ Any links for those 1930 and 1980 references? $\endgroup$– MitchCommented Dec 3, 2019 at 21:59
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong (naive) independence assumptions.
Bayes' theorem was named after the Reverend Thomas Bayes (1702–61), who studied how to compute a distribution for the probability parameter of a binomial distribution. After Bayes' death, his friend Richard Price edited and presented this work in 1763, as An Essay towards solving a Problem in the Doctrine of Chances.
So it is safe to say that Bayes classifiers have been around since the 2nd half of the 18th century.
Especially as Stephen Stigler suggested (in 1983, Stephen M. Stigler, "Who Discovered Bayes' Theorem?" The American Statistician 37(4):290–296) that Bayes' theorem was discovered by Nicholas Saunderson some time before Bayes. On the other hand Edwards (1986) disputed that interpretation (in 1986, A. W. F. Edwards, "Is the Reference in Hartley (1749) to Bayesian Inference?", The American Statistician 40(2):109–110).
Which takes us back to the safe assumption of "2nd half of the 18th century" again, as a naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem... which makes it "naive" is that it comes with strong (naive) independence assumptions. But practically, it's the same theorem.
-
$\begingroup$ Is that all there is to naive Bayes classification? Isn't there a little more machinery involved for multiple classes? Any example of that before the 1930's? $\endgroup$– MitchCommented Dec 3, 2019 at 21:59
$\begingroup$
$\endgroup$
1
I have seen the following paper cited before for Naive Bayes:
Hand, D. J., & Yu, K. (2001). Idiot's Bayes—not so stupid after all?. International statistical review, 69(3), 385-398.
It is a bit of a review and discussion of the topic.
-
$\begingroup$ Can you add to your answer here what that article says about the timing of the first use of Naive Bayes as a classifier (and also if the name itself has a later origin)? $\endgroup$– MitchCommented Feb 10, 2020 at 20:55
$\begingroup$
$\endgroup$
1
The question is asking about the procedure currently known as naive Bayes (with various alternate spellings). This is an entirely distinct issue from the provenance and naming of Bayes' Theorem.
Checking the paper mentioned in dpritch's answer, they give a reference to the paper
Kononenko, I. (1990). Comparison of inductive and naïve Bayesian learning approaches to a In Current trends in knowledge acquisition, Eds. B. Wielinga et al. Amsterdam Press IOS
as the first mention of the term 'naïve Bayes'. Online searches show no mention of minor variations of that term before then.
So 1990, is most likely the first appearance of the -term- 'Naive Bayes'.
As to when the method first appeared, under any name, is a bit more murky. Russell and Norvig, Artificial Intelligence: a Modern Approach gives in its notes that the procedure had been used in pattern recognition since the 50's and explicitly in
Robertson and Spark Jones (1976), Relevance weighting of search terms, Journal American Society for Information Science 27, 129-146.
(but without naming it as naive Bayes).
So TLDL, classification using multiple factors 'naively' or out of expedience assumed to be independent was probably started in the 50's, but that procedure didn't get the name 'Naive Bayes' until Kononenko 1990 which has references to prior art).
-
$\begingroup$ Good answer! The currently accepted answer doesn't even address the question $\endgroup$ Commented Feb 22 at 2:04