0
$\begingroup$

I know that Laplace smoothing results in a high bias of Naive Bayes. If the value of the smoothing parameter (alpha) is large, then the probability distribution will be uniform for all the features. And Naive Bayes classifies based on the values of the likelihood of conditional probabilities. So the model will now classify based on the class label resulting in a high bias. Now, my question is can a Naive Bayes model overfit the data? If so How?

$\endgroup$
0
$\begingroup$

In general, overfitting is not something you should worry that much with naive Bayes. It’s more likely to underfit. Naive Bayes is a fairly simple algorithm, making a strong assumption of independence between the features, so it would be biased and less flexible, hence less likely to overfit. But it is possible. For example, say that in your training set you have a variable of variables that have very high (or low) probabilities influencing the predictions and the probabilities differ from what you’d see outside training data. The model would be accurate at training time but not prediction time, hence would overfit.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.