3
$\begingroup$

I was reading the Naive Bayes article on Wikipedia and I read that, In Naive Bayes, the naive assumption that Naive Bayes make is "each feature is conditionally independent of every other feature, given the category Ck". So, I was wondering if 'Conditional Independence' means there should be no multicollinearity among features?

$\endgroup$
4
$\begingroup$

No, it does not imply lack of multicolineaity. Suppose $X$ and $Y$ are normal with variance 0.001, and have mean $\mu = 100$ if $Z = 1$ and mean $\mu = -100$ if $Z = 0$, but are conditionally independent given $Z$. There is still extreme multicolineairty here, with the correlation being very close to $1$.

$\endgroup$
  • $\begingroup$ So, What could be an intuitive explanation of conditional independence in Naive Bayes? $\endgroup$ – Akash Dubey Sep 25 '18 at 14:27
  • $\begingroup$ @AkashDubey I fail to see how multicollinearity has anything to do with conditional independence or Naive Bayes here. Are you running into this problem with a data analysis? As an example, two factors can have 0 prevalence but be conditionally independent, as such both terms are nominally multicollinear, but not necessarily dependent. $\endgroup$ – AdamO Sep 25 '18 at 15:03
  • 1
    $\begingroup$ @AkashDubey conditional independence states that if you knew $Z$ in this example, then they are independent. This implies that at every fixed level of $Z$ in the above example, you will not have multicolinearity. But you can still have multicolinearity if you ignore $Z$. $\endgroup$ – guy Sep 25 '18 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.