One of the main assumptions of the naive Bayes model is that the features are independent. This allows probabilities to be estimated. However, often times it is understood that this assumption doesn't hold in the data. Naive Bayes is, however, still used for applications like text classification because it still provides "good results."

My question is:

1) Is there any way to check how well the assumption holds? I.e., can I test for conditional independence?

2) Is there a way to determine what "good results" are. I.e., are "good results" from a naive Bayes model, vs any other model, based on things like accuracy/how well the model cross validates on training data?


2 Answers 2


You don’t check the assumption of independence when using Naive Bayes, you don’t even expect it to hold. The textbook example of using Naive Bayes is spam detection with text data, we don’t use it in such cases because we assume that words in human language appear in random combinations, that are independent, but because the algorithm has proven to work in such setting. Moreover, it would be hard if not impossible for you to find data where features are independent, usually there exists some, non-zero degree of dependence.


1) There is no single answer to this question. A good metric for linear dependence is a correlation matrix. i.e. cor(x) in R. If the features are linearly independent, they are uncorrelated, hence the non-diagonal entries of the correlation matrix should be close to zero. See here for a discussion of why naive bayes does well even when its assumptions are violated.

2) Again, there is no single answer to this question. The most common way to compare machine learning models is to see how well they do on held out data and the most common metric is AUC.

  • $\begingroup$ Correlation measures only pairwise relationships between variables, while in native Bayes we are talking about joint distribution of all the variables. Also correlation measures only linear relationships. So it does not help much. $\endgroup$
    – Tim
    Jul 31, 2020 at 15:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.