# how to minimize the probability of error in a Bayes decision rule

According to the Bayes decision rule for a 2 class classification problem:

$d(x) = w_1 : P(w_1 |x) ≥ P(w_2|x)$

And $P(error|x) = min[P(w_1 |x), P(w_2|x)]$

where $P(w_i |x) = p(x|w_i) * P(w_i)$

1. By simply following this decision rule $d(x)$, we minimize the probability of error? There is no complicated way or additional formulas for minimizing error?
2. In case we want to minimize this error further how do we do it? Increase the number of samples perhaps? Or maybe increase the dimension of feature space i.e., use more features, thereby changing the likelihood $p(x|w_i)$, or combination of both.

Given some underlying model for the data, you cannot do better than the Bayes optimal rule (hence the "optimal"). It's actually fairly easy to see. Suppose you have an observation $x$ and want to classify it into one of $k$ groups. Then whatever your decision might be, if you are assigning it to a class other than the most likely one, you will be wrong more often than the Bayes classifier. Simply collecting more data or something else can't change this.