According to the Bayes decision rule for a 2 class classification problem:

$d(x) = w_1 : P(w_1 |x) ≥ P(w_2|x) $

And $P(error|x) = min[P(w_1 |x), P(w_2|x)]$

where $P(w_i |x) = p(x|w_i) * P(w_i)$

  1. By simply following this decision rule $d(x)$, we minimize the probability of error? There is no complicated way or additional formulas for minimizing error?
  2. In case we want to minimize this error further how do we do it? Increase the number of samples perhaps? Or maybe increase the dimension of feature space i.e., use more features, thereby changing the likelihood $p(x|w_i)$, or combination of both.

Given some underlying model for the data, you cannot do better than the Bayes optimal rule (hence the "optimal"). It's actually fairly easy to see. Suppose you have an observation $x$ and want to classify it into one of $k$ groups. Then whatever your decision might be, if you are assigning it to a class other than the most likely one, you will be wrong more often than the Bayes classifier. Simply collecting more data or something else can't change this.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.