# On the definition of the Bayes Classifier

Let $(X,Y): \Omega \to (\mathbb{R}^d, F)$, with $F = \left\{ 1, \dots, n \right\}$ be a random variable. According to Wikipedia the Bayes classifier is $$C_B(x) = \arg\, \max \mathbb{P}(Y= i \,|\, X=x),$$ with $x \in \mathbb{R}^d$. Now I understand the idea, "pick the most likely one". However, if $X$ has a density, then $X=x$ is a zero probability event. How should conditioning on this event be understood?

Of course you can define something like $$\mathbb{P}(Y=i \,|\, X=x) = \lim \limits_{\varepsilon \to 0} \mathbb{P}(Y=i \, | \, X \in B(x, \varepsilon)),$$ where $B(x,\varepsilon)$ is a ball of radius $\varepsilon > 0$ around $x$, but I'm unable to find a reference for this.

• While $\mathbb{P}(X=x)=0$, the conditional distribution of $Y$ given $X=x$ is uniquely defined almost everywhere. Commented Nov 15, 2016 at 13:37
• $P(Y=i \vert X=x)$ is to be interpreted as having observed $x$, what is the probability that it comes from class $i$. It is computed from $f(x\vert Y=i)p(Y=i)$, where we assume that the class conditional densities $f(x\vert Y=i)$ and prior probs $p(Y=i)$ are known in advance. Commented Nov 15, 2016 at 14:02
• Recent related question: stats.stackexchange.com/questions/246009/…
– Tim
Commented Nov 15, 2016 at 14:57