# Bayes classifier?

I'm trying to understand the Bayes Classifier. I don't really understand its purpose or how to apply it, but I think I understand the parts of the formula:

$$P(Y = j \mid X = x_{0})$$

If I'm correct, it's asking for the largest probability, depending on one of two conditions. If $Y$ is equal to some class $j$, or if $X$ is some data point $x_{0}$.

How would I compute this with a data set $(x, y)$ where $x$ is just a number between 1 and 100, and $y$ is one of two classes ("blue" or "orange"), e.g. (5, "blue"), (51, "orange")? Does this data set even work to apply the classifier or should I consider making a new data set?

Sorry if it's a silly question, I'm out of touch with my statistics. Some pseudocode would be terrific, but I'll be applying this in R. I'm not interested in the R function to complete this. Some regular guidance with good ol' math would be great as well.

Thank you for any help!

• Please write a more informative title. – Franck Dernoncourt Feb 9 '17 at 20:04

## 2 Answers

Interpret the formula as follows: What is the probability of Y being equal to j, when we know X = x0. So in your dataset, the bayes classifier is effectively computing probabilities of achieving blue or orange when you define the value of x. If in your data, when x is greater than 75, if 90% of the balls are orange, then the classifier will choose orange whenever this happens.

This is a very "non-technical" explanation and I hope it helps you understand the basic idea.

So when someone chooses to use a Bayes classifier (or any other classifier for that matter) you use it to predict categorical outcomes based on one or more input variables that may be continuous or categorical.

• Hey! Thanks for the answer. I think I'm getting a better idea of it. You're saying that the assignment of "blue" or "orange" to my points in my dataset shouldn't be random, but while generating the data I should give a % chance of the class being assigned? As you say, like a 90% chance of "orange" after $x = 75$? That means that my formula, using that example, could look something like this? $P(Y = orange|X = 75) = 0.9$ – Dan Sep 30 '16 at 6:49
• Yes, usually the data is not generated by you but is produced by some system. For example people who buy a car in response to an advertisement. Using the bayes' classifier you will be able to predict who will most likely buy your car given some knowledge about the individual. – Arun Jose Sep 30 '16 at 8:18

The Bayes classifier is the one that classifies according to the most likely category given the predictor $x$, i.e.,

$$\text{arg max}_j P(Y = j \mid X = x) .$$

Since these "true" probabilities are essentially never known, the Bayes classifier is more a theoretical concept and not something that you can actually use in practice. However, it's a helpful idea when doing simulation studies where you generate the data yourself and therefore know the probabilities. This allows you to compare a given classification rule to the Bayes classifier which has the lowest error rate among all classifiers.