For instance if we have an outcome with 3 classes A,B, and C. Assuming $X$~$Uniform(0,1)$ and $Y$ is as follows

If $ x > 0.6$

$P(Y = A) = 0.4$

$P(Y = B) = 0.3$

$P(Y = C) = 0.3$

Alternatively, if $ x <= 0.6$

$P(Y = A) = 0$

$P(Y = B) = 0.2$

$P(Y = C) = 0.8$

What is the Bayes Classifier and Bayes Error and how does that change with different distributions for x, say if $X$~$Normal(0,1)$


The Bayes classifier chooses the class with maximum posterior probability, i.e. $P(Y=c|X=x)$ and you have all you need; in other words, you have the posterior probabilities for each possible $x$. For example, if $X=0.2$, $P(Y=C|X=0.2)$ is the maximum, and the class is $C$.

The Bayes error you make is the error when you decide incorrectly, based on the Bayes decision rule. For example, $P(X>0.6)=0.4$ if $X\sim U[0,1]$, and if $X>0.6$, we choose class $A$ as our decision. There is $0.3+0.3=0.6$ probability of the correct class being not $A$. Merging the other case yields to following formula, just waiting for the values to be substituted:

$$P(\epsilon)=P(\epsilon|X>0.6)P(X>0.6)+P(\epsilon|X\leq 0.6)P(X\leq 0.6)$$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.