4
$\begingroup$

I'm trying to understand the Bayes Classifier. I don't really understand its purpose or how to apply it, but I think I understand the parts of the formula:

$$P(Y = j \mid X = x_{0})$$

If I'm correct, it's asking for the largest probability, depending on one of two conditions. If $Y$ is equal to some class $j$, or if $X$ is some data point $x_{0}$.

How would I compute this with a data set $(x, y)$ where $x$ is just a number between 1 and 100, and $y$ is one of two classes ("blue" or "orange"), e.g. (5, "blue"), (51, "orange")? Does this data set even work to apply the classifier or should I consider making a new data set?

Sorry if it's a silly question, I'm out of touch with my statistics. Some pseudocode would be terrific, but I'll be applying this in R. I'm not interested in the R function to complete this. Some regular guidance with good ol' math would be great as well.

Thank you for any help!

Improve this question
$\endgroup$
1
  • $\begingroup$ Please write a more informative title. $\endgroup$
    – Franck Dernoncourt
    Commented Feb 9, 2017 at 20:04

2 Answers 2

Reset to default
1
$\begingroup$

Interpret the formula as follows: What is the probability of Y being equal to j, when we know X = x0. So in your dataset, the bayes classifier is effectively computing probabilities of achieving blue or orange when you define the value of x. If in your data, when x is greater than 75, if 90% of the balls are orange, then the classifier will choose orange whenever this happens.

This is a very "non-technical" explanation and I hope it helps you understand the basic idea.

So when someone chooses to use a Bayes classifier (or any other classifier for that matter) you use it to predict categorical outcomes based on one or more input variables that may be continuous or categorical.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hey! Thanks for the answer. I think I'm getting a better idea of it. You're saying that the assignment of "blue" or "orange" to my points in my dataset shouldn't be random, but while generating the data I should give a % chance of the class being assigned? As you say, like a 90% chance of "orange" after $x = 75$? That means that my formula, using that example, could look something like this? $P(Y = orange|X = 75) = 0.9$ $\endgroup$
    – Dan
    Commented Sep 30, 2016 at 6:49
  • $\begingroup$ Yes, usually the data is not generated by you but is produced by some system. For example people who buy a car in response to an advertisement. Using the bayes' classifier you will be able to predict who will most likely buy your car given some knowledge about the individual. $\endgroup$
    – Arun Jose
    Commented Sep 30, 2016 at 8:18
10
$\begingroup$

The Bayes classifier is the one that classifies according to the most likely category given the predictor $x$, i.e.,

$$ \text{arg max}_j P(Y = j \mid X = x) . $$

Since these "true" probabilities are essentially never known, the Bayes classifier is more a theoretical concept and not something that you can actually use in practice. However, it's a helpful idea when doing simulation studies where you generate the data yourself and therefore know the probabilities. This allows you to compare a given classification rule to the Bayes classifier which has the lowest error rate among all classifiers.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.