# What is the extension of Naive Bayes that breaks the conditonal independence?

I had the idea that we can overcome the conditional independence of features within Naive Bayes classification by assuming that we have latent (hidden) sub-classes. Let me explain.

For example, if we have this features vector

(1, 1, 0, 1, 0)


in Naive Bayes we would calculate probability of this vector given a class like this:

$$p_1 \cdot p_2 \cdot (1 - p_3) \cdot p_4 \cdot (1 - p_5),$$

where $$p_i$$ is probability of feature to be present (value 1) given the class.

However, as a generalization, we can say that within a given class we have two sub-classes. In this case probability of the above given vector will be calculated as:

$$p_1 \cdot p_2 \cdot (1 - p_3) \cdot p_4 \cdot (1 - p_5) \cdot \nu_1 + p'_1 \cdot p'_2 \cdot (1 - p'_3) \cdot p'_4 \cdot (1 - p'_5) \cdot \nu_2,$$

where $$p_{i}$$ ans $$p'_{i}$$ are probabilities of the feature $$i$$ to be present given the first and the second sub-classes, respectively. $$\nu_j$$ are the probabilities o the two sub-classes.

I assume that this method was already introduced and considered in details. So, my question is what is the name of the method and what is efficient way to calculate its parameters based on a given data set?

• Why not just use logistic regression Commented Jun 14, 2023 at 14:23
• @seanv507 because logistic regression assumes very simple (linear) dependency. Commented Jun 15, 2023 at 9:53
• Linear in parameters doesn't mean you can't add arbitrary feature transformations: interactions, logs etc Commented Jun 15, 2023 at 16:30