3
$\begingroup$

Show that the bayes classifier will achieve the best error rate, defined as:

$$ E(f) = \int \int \mathbb{I}(y = f(x)) \cdot p(x, y) dxdy $$

where $$f(x)$$ is the classifier, and $$p(x, y)$$ is the intrinsic data distribution.

I just want to get some help as to how to proceed.

$\endgroup$
6
  • $\begingroup$ You probably need to add the self-study tag to this, and read the wiki for said tag. Your question is also not well posed, as you do not define "error rate", and the successful resolution of your query almost certainly depends on the correct interpretation of that term (there are many ways to define "error rate"). $\endgroup$ Sep 26, 2015 at 14:49
  • $\begingroup$ The definition of error rate is given in the question $\endgroup$ Sep 26, 2015 at 15:01
  • $\begingroup$ Ahh, I read incorrectly, my humble apologies - I am sorry for my poor reading comprehension. Please add the self-study tag none-the-less. $\endgroup$ Sep 26, 2015 at 15:05
  • $\begingroup$ How is $l$ defined, in the integral? $\endgroup$ Sep 29, 2015 at 15:29
  • $\begingroup$ What do u mean by l ? $\endgroup$ Sep 29, 2015 at 18:33

2 Answers 2

1
$\begingroup$

If by Bayes classifier you mean a maximum a posteriori probability (MAP or MAPP) decision rule, then for discrete random variables, an intuitive explanation can be found here. More generally, the fact that the minimum probability of error decision rule is the one that decides that $H_i$ is the true hypothesis if $$\pi_if_i(x) > \pi_{1-i}f_{1-i}(x),$$ is easily derived (see, e.g. here for details). The rule above can also be expressed as $$L(x) = \frac{f_1(x)}{f_0(x)} ~~\begin{array}{c}H_1\\ \gtrless\\H_0\end{array}~~\frac{\pi_0}{\pi_1}$$ which can be recognized as a likelihood ratio test with the Bayesian threshold.

$\endgroup$
0
$\begingroup$

Let $f$ be the Bayes classifier and let $g$ be another classifier. Note that for any $x$,

$$ \int \mathbb{I}[y=f(x)]\cdot p(y|x)dy = p(f(x)|x) \geq p(g(x)|x) = \int \mathbb{I}[y=g(x)]\cdot p(y|x)dy, $$ by the definition of $f$. Once we see that $$ E(f) = \int p(f(x)|x) \cdot p(x) dx, $$ we are done.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.