Dependent Features and Naive Bayes

Question

Naive Bayes assumes that the features given their classes are independent, and hence :

$$P(y~|~x_1, \ldots, x_n)= \frac{P(y)P(x_1,\ldots, x_n~|~ y) }{P(x_1,\ldots,x_n)}$$ Will become :

$$ P(y~|~ x_1,\ldots,x_n) =\frac{P(y)\prod_{i=1}^n P(x_i~| ~y)}{P(x_1,\ldots,x_n)}$$ That is, due to the assumption that the features conditioned to their class are independent, then we have multiplied each feature conditioned to its class by the other. My question is, if the features are dependent then what is the right way to calculated the features conditioned to their class ?

Answer 1

Even under the Naive Bayes condition, features can be dependent. It is conditioned on the class that the features have to be independent. And if the class conditionals are not independent, your first formula cannot be transformed into the convenient second one. I.e., $p(x_1, \ldots, x_n | y)$ cannot be factorized and you have to try some of the many other classification techniques.