# What does it mean "Disadvantage of Naive Bayes Classifier: strong feature independence"?

It is told that

the most important disadvantage of Naive Bayes is that it has strong feature independence assumptions.

Can someone please explain this more elaborately?

• The Naive Bayes classifier assumes that all variables are conditionally independent given the outcome. This assumption rarely holds in practice. Nov 23, 2015 at 1:17

$\newcommand{\vect}[1]{\mathbf{#1}}$

The Naive Bayesian classifier explores the idea of maximizing posterior probability that given tuple $\vect{X} = (x_1, x_2, \dots, x_N)$ belongs to the class $C_{i}$, i.e. maximizing $P(C_{i}|\vect{X})$.

By Bayes' theorem $$P(C_{i}|\vect{X}) = \frac{P(\vect{X}|C_{i})P(C_{i})}{P(\vect{X})}$$

$P(\vect{X})$ is constant for the classes, and if we don't have any prior for $P(C_{i})$, we assume $P(C_{i}) = P(C_{j})$.

So we have only $P(\vect{X}|C_{i})$ to compute for all the training data. And this is there the "Naive" assumption is made: we assume that there is no dependence relationships between attributes.

That means $$P(\vect{X}|C_{i}) = \prod_{k=1}^{N} P(x_k|C_{i})$$

So now we can estimate the probabilities $P(x_1|C_{i})$, $\dots$, which is very easy compared to estimating $P(\vect{X}|C_{i})$.

The price paid for this easiness is the class-condition independence assumption made above, which is not always a true.