Skip to main content
Notice removed Draw attention by CommunityBot
Bounty Ended with no winning answer by CommunityBot
Tweeted twitter.com/StackStats/status/1251797714796531714
Notice added Draw attention by Sobi
Bounty Started worth 50 reputation by Sobi
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
edited tags
Link
Tim
  • 141.1k
  • 26
  • 270
  • 512
Source Link
Sobi
  • 2.3k
  • 16
  • 23

Why classifiers report the class with maximum posterior probability as the predicted class?

When we train a classifier to predict $y \in \{1, \dots, K\}$ given an input $x$, classification is done by reporting the class with the highest posterior probability as the prediction; that is: $$ \hat{y}(x) = \arg\max_{\hspace{-0.75cm} y \in \{1, \dots, K\}} P(y | x) $$ The justification for such a prediction rule, as far as I know, it that this gives the Bayes optimal classifier.

This makes sense to me only if $P(y | x)$ matches the true distribution of the data exactly. However, $P(y | x)$ is only an estimated model that we fit to the training data, probably after making lots of incorrect assumptions, and most likely does not perfectly match the distribution of the actual data.

My question is, does the Bayes optimal classification rule still applies to estimated models, or does it only hold for the true underlying distribution of the data.