Are posterior probabilities from a Naive Bayes classifier reliable? I have read that the posterior probabilities of Naive Bayes classifiers are unreliable. Is this true? and if so, in what sense, and why?
Specifically, I am interested to know if the probabilities can be used as some measure of confidence of the predictions. For example, can I say that higher posteriors are more likely to be correct, or use a threshold to say that if the posterior is below some value then it is low-confidence.
 A: Sorry, too long for a comment.
Very interesting. The quote

although naive Bayes is known as a decent classifier, it is known to be a bad >estimator, so the probability outputs from predict_proba() are not to be taken >too seriously 

is taken from this paper that in turn is citing Bennett and Monti and Cooper. I don't think Bennett is very convincing since he only claims that "many people" think that NB produces estimates too close to 1 and 0, but gives no proof. He offers as additional evidence the fact that for long documents, there is a tendency to push values to 0 or 1, even when they are completely wrong. Monti and Cooper's paper is a bit tricky because they're comparing finite mixture models (FM), finite mixture augmented Naive-Bayes model (FAN) and Naive Bayes, so they tend to describe more about this comparison than to describe in detail why Naive Bayes are poor estimators. 
On the other hand, you can find the same claim for decision trees and SVMs, so maybe people think that a given classifier gives poor estimates when it is not an issue intrinsically associated to that classifier. 
I think we simply need better references and better evidence. I hope other people can provide more information.
