Doubt in bayes classifier error calculation

Question

I have recently started machine learning on my own. I started reading Duda art and start book. That author says that Bayes classifier has a min error. He calculates

$$\begin{equation} P(error|x)=\begin{cases} P(\omega_1 |x), & \text{if we decide as $\omega_2$}.\\ P(\omega_2|x), & \text{if we decide as $\omega_1$}. \end{cases} \end{equation} $$

How did he write this probability of error? I know we decide $\omega_1$ if $P(\omega1 |x)>P(\omega_2|x)$ and vice versa. From this he calculates

$$P(error)=\int_X P(error |x)P(x) dx = \int_X min\{P(\omega_1 |x), P(\omega_2 |x) \} P(x) dx$$

and this will be minimum. This part I understood. But how $P(error|x)$ equation is writtten I did not understand. Please clarify.

Another doubt:

In case of multi-classes(instead of 2 classes), we are deciding to which class X(d-dimensional feature vector) belongs to based on for which class conditional risk$R(\alpha_i |x)$ is minimum instead of for which class posterior probability $P(\omega_i |x)$ is maximum. Why are we focussing on minimum condiitonal risk rather than maximum posterior probability(this is what is bayes classifier in real sense right?). Kindly explain

Answer 1

I didn't look up the book, so I will guess the meaning of notation from your posting.

1) There are two possible decisions $\omega_1$ and $\omega_2$. There are two possible truths, $\omega_1$ and $\omega_2$. You're committing an error if you decide wrongly, so you decide $\omega_1$ when in fact the truth is $\omega_2$, or the other way round. The event "error" is therefore "either decide $\omega_1$ if it's in fact $\omega_2$, or decide $\omega_2$ if it's in fact $\omega_1$." $P(error|x)$ is the probability of "error" given $x$, which is the probability of what I wrote down above, i.e., $P(\omega_2|x)$ (meaning the probability that $\omega_2$ is true knowing $x$) if you decide $\omega_1$, or $P(\omega_1|x)$ if you decide $\omega_2$ (this holds generally, not specifically for the Bayes classifier). I'm assuming here that $x$ determines the decision, so that decision probabilities given $x$ don't need to be incorporated in the probability computation.

2) Risk minimisation allows for more flexibility than posterior probability maximisation. If the loss for both errors is the same, risk minimisation is the same as posterior probability maximisation. However, if you lose much more in case of one error than in case of the other (let's say in case you decide $\omega_1$ if in fact $\omega_2$ is true), you may change your decision rule so that you decide $\omega_2$ somewhat more often than the posterior probability would suggest, because then your probability is higher to avoid the error that costs you more, even though you'll have to pay for that by increasing the probability for the other possible error.

(PS: I do realise that you asked this question for more than two classes, however there the same logic applies... even for two classes you may be interested in risk minimisation.)