Skip to main content
Cross Validated

Return to Answer

added 10 characters in body
Source Link
Tim
Tim
  • 141.2k
  • 26
  • 270
  • 512

Quoting my other answer:

Naive Bayes algorithm makes the "naive" assumption, that the features are (conditionally) independent, so by definition of independence

$$ p(x_1, x_2, \dots, x_k \mid y) = \prod_{i=1}^k p(x_i | y) $$

[...] the estimate of conditional probability would be accurate if the assumption of independence holds (and in real life it almost never does).

Saying it differently, the probabilities predicted by naive Bayes algorithm are not well callibrated, so they do not reflect the true probabilities.

On another hand, while the probabilities themselves are not good estimates of the conditional probabilities, they do give you information about relative ordering of the classes. If one class has higher predicted probability, thenas compared to another, then you should prefer the former as your classification.

Referring to your question, if for classes $c_1, c_2, \dots, c_k$, the algorithm predicted probabilities $0.3, 0.2, \dots, 0.1$, and other source of information tell you $c_3, c_4, \dots, c_k$ are impossible (i.e. have zero probability), then yes, you should prefer $c_1$ over $c_2$, since the predicted probabilities tell you something about relative ordering of the classes. You can find formal argument for this in here.

Quoting my other answer:

Naive Bayes algorithm makes the "naive" assumption, that the features are (conditionally) independent, so by definition of independence

$$ p(x_1, x_2, \dots, x_k \mid y) = \prod_{i=1}^k p(x_i | y) $$

[...] the estimate of conditional probability would be accurate if the assumption of independence holds (and in real life it almost never does).

Saying it differently, the probabilities predicted by naive Bayes algorithm are not well callibrated, so they do not reflect the true probabilities.

On another hand, while the probabilities themselves are not good estimates of the conditional probabilities, they do give you information about relative ordering of the classes. If one class has higher predicted probability, then another, then you should prefer the former as your classification.

Referring to your question, if for classes $c_1, c_2, \dots, c_k$, the algorithm predicted probabilities $0.3, 0.2, \dots, 0.1$, and other source of information tell you $c_3, c_4, \dots, c_k$ are impossible (i.e. have zero probability), then yes, you should prefer $c_1$ over $c_2$, since the predicted probabilities tell you something about relative ordering of the classes. You can find formal argument for this in here.

Quoting my other answer:

Naive Bayes algorithm makes the "naive" assumption, that the features are (conditionally) independent, so by definition of independence

$$ p(x_1, x_2, \dots, x_k \mid y) = \prod_{i=1}^k p(x_i | y) $$

[...] the estimate of conditional probability would be accurate if the assumption of independence holds (and in real life it almost never does).

Saying it differently, the probabilities predicted by naive Bayes algorithm are not well callibrated, so they do not reflect the true probabilities.

On another hand, while the probabilities themselves are not good estimates of the conditional probabilities, they do give you information about relative ordering of the classes. If one class has higher predicted probability, as compared to another, then you should prefer the former as your classification.

Referring to your question, if for classes $c_1, c_2, \dots, c_k$, the algorithm predicted probabilities $0.3, 0.2, \dots, 0.1$, and other source of information tell you $c_3, c_4, \dots, c_k$ are impossible (i.e. have zero probability), then yes, you should prefer $c_1$ over $c_2$, since the predicted probabilities tell you something about relative ordering of the classes. You can find formal argument for this in here.

Source Link
Tim
Tim
  • 141.2k
  • 26
  • 270
  • 512

Quoting my other answer:

Naive Bayes algorithm makes the "naive" assumption, that the features are (conditionally) independent, so by definition of independence

$$ p(x_1, x_2, \dots, x_k \mid y) = \prod_{i=1}^k p(x_i | y) $$

[...] the estimate of conditional probability would be accurate if the assumption of independence holds (and in real life it almost never does).

Saying it differently, the probabilities predicted by naive Bayes algorithm are not well callibrated, so they do not reflect the true probabilities.

On another hand, while the probabilities themselves are not good estimates of the conditional probabilities, they do give you information about relative ordering of the classes. If one class has higher predicted probability, then another, then you should prefer the former as your classification.

Referring to your question, if for classes $c_1, c_2, \dots, c_k$, the algorithm predicted probabilities $0.3, 0.2, \dots, 0.1$, and other source of information tell you $c_3, c_4, \dots, c_k$ are impossible (i.e. have zero probability), then yes, you should prefer $c_1$ over $c_2$, since the predicted probabilities tell you something about relative ordering of the classes. You can find formal argument for this in here.