Repeating the question

What advantage does Naive Bayes have over "not naive" Bayes? Considering the fact that the assumption about conditional independence is often violated, why do we make it?

As pretty much any source on the internet states, assumption about conditional independence between features rarely holds. To make things more concrete, consider following example

enter image description here


$$Y := \{\text{boys}\}$$ $$X_1 := \{\text{people that have big muscles}\}$$ $$X_2 := \{\text{people that have short hair}\}$$

Then $P(X_2 \mid Y \cap X_1) = \frac{1}{2}$ but $P(X_2 \mid Y ) = \frac{2}{3}$, implying that $X_1$ and $X_2$ are not conditionally independent.

So repeating the question:

Why do we assume conditional independence when using Bayesian classifier? What advantages does Naive Bayes have over the "not naive" Bayes (i.e algorithm that doesn't assume cond. independence)?

  • 3
    $\begingroup$ a) its computationally much cheaper - this is applied mainly to huge data sets. b) even if the probabilities are wrong, ranking/classification is often good. $\endgroup$ – seanv507 Apr 24 at 18:14

The primary advantage is that the Naive Bayes classifier is a simpler model, and in most cases requires less training data to perform well than a more complex Bayesian network.

This question is worth reading for further discussion of Naive Bayes vs Bayesian networks: https://stackoverflow.com/questions/12298150/what-is-the-difference-between-a-bayesian-network-and-a-naive-bayes-classifier?rq=1

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.