# Bayesian Naive Classifier with 0 variance

I am experimenting with the Bayesian Naive Classifier and I have come across a situation for which I wasn't able to find a solution on the net.

Suppose you have a dataset with a few attributes that takes real values. The values of the observations varies around a mean as they typically do -- except for a particular attribute and for a particular class where all observations have the same value. For that particular attribute-class subset, the variance will be 0 and the probability density for such a distribution will be infinity for any 'x'.

How should a BNC handle such situations? Should we discard an attribute if it has a degenerate (zero variance) distribution for any class even if it has a normal distribution otherwise?

Edit:

I'd rather not ignore them if possible. They may have a lot of information, just not for the class for which variance is 0. I'll try to explain myself:

Suppose you have a bowl of cookies, some of them were made by a machine and the others were handmade and you want to classify them (as well as eating them). One of the attributes you use to classify is the diameter of the cookie. A handmade cookie will have a good variance while the machine-made ones all have the same (0 variance).

Say all machine-made cookie are all 2.5 inches in diameter. I agree that picking a 2.5 inch cookie at random will give us no useful information; but picking a 3 inch cookie is a pretty strong indication that is was handmade.

In the case of a 0 variance distribution, I was thinking making the Probability Density function return a 1.0 (neutral) when 'x' is the mean and 0.0 when it's not. But I'm not experienced enough in statistics to understand the possible side-effects of this.

As for where the formula prior * x/(prior * x+1-prior) comes from:

The number of homemade cookies with a machine mean is proportional to the prior probability times the conditional probability, prior * x. For machine cookies, the conditional probability is 1, so it's just the prior probability of machine, which is 1-(prior probability of homemade).

Often, when learning about conditional probabilities, it's easier to work with proportions rather than percentages (that is, "10 out of 100" is more concrete than "10%"). If you're having trouble following an argument, putting in integers and taking successively smaller portion of the total as more conditions are added can help.

Now, again, I introduced the 1000 just so that we would have integer cookies and the math would be easier to follow; the 1000 should of course cancel out, so if you take the formula

prior * x /(prior * x+1-prior)

and substitute in prior = 20%, x = 10%, you get

.2*.1/(.2*.1+.8) = .02/.82 = .024

• Thanks a lot @Accumulation. I would be interested in knowing why the probability of being a homemade cookie should be prior * x/(prior * x+1-prior). Please can you tell me where this comes from? (Sorry if it's a basic stats question, but I'm pretty much a beginner) Apr 13, 2018 at 21:17

Within statistics, all is uncertain, also the class membership of each pattern. Probabilities that are purely 0 and 1, that indicates a deterministic mechanism, for which statistics was not developed. As soon as uncertainty comes into play, a deterministic paradigm is left and it becomes statistics and probability theory.

When you can with 100% certainty (on a very large, representative dataset) classify all patterns correctly, you get into the deterministic situation. Boolean algebra with truth-predicates and (in)equalities are sufficient to represent the classification rule.

• Thanks Match Maker. If I understand correctly, you say that in situations like the cookie problem, I don't need a classifier; a if() would do just fine. Apr 13, 2018 at 18:50
• Correct - that is the point I wanted to make. Apr 13, 2018 at 19:20