Bayes classifier? I'm trying to understand the Bayes Classifier. I don't really understand its purpose or how to apply it, but I think I understand the parts of the formula:
$$P(Y = j \mid X = x_{0})$$
If I'm correct, it's asking for the largest probability, depending on one of two conditions. If $Y$ is equal to some class $j$, or if $X$ is some data point $x_{0}$.
How would I compute this with a data set $(x, y)$ where $x$ is just a number between 1 and 100, and $y$ is one of two classes ("blue" or "orange"), e.g. (5, "blue"), (51, "orange")? Does this data set even work to apply the classifier or should I consider making a new data set?
Sorry if it's a silly question, I'm out of touch with my statistics. Some pseudocode would be terrific, but I'll be applying this in R. I'm not interested in the R function to complete this. Some regular guidance with good ol' math would be great as well.
Thank you for any help!
 A: The Bayes classifier is the one that classifies according to the most likely category given the predictor $x$, i.e.,
$$
\text{arg max}_j P(Y = j \mid X = x) .
$$
Since these "true" probabilities are essentially never known, the Bayes classifier is more a theoretical concept and not something that you can actually use in practice.  However, it's a helpful idea when doing simulation studies where you generate the data yourself and therefore know the probabilities.  This allows you to compare a given classification rule to the Bayes classifier which has the lowest error rate among all classifiers.
A: Interpret the formula as follows: What is the probability of Y being equal to j, when we know X = x0. So in your dataset, the bayes classifier is effectively computing probabilities of achieving blue or orange when you define the value of x. If in your data, when x is greater than 75, if 90% of the balls are orange, then the classifier will choose orange whenever this happens.
This is a very "non-technical" explanation and I hope it helps you understand the basic idea.
So when someone chooses to use a Bayes classifier (or any other classifier for that matter) you use it to predict categorical outcomes based on one or more input variables that may be continuous or categorical.
