A Probability distribution value exceeding 1 is OK?

Question

On the Wikipedia page about naive bayes classifiers here there is this line "P(height|male) = 1.5789 (A probability distribution over 1 is OK. It is the area under the bell curve that is equal to 1.)" How can a value > 1 be OK? I thought all probability values were expressed in the range 0 <= p <= 1. Furthermore, given that it is possible to have such a value, how is that value obtained in the example shown on the page?

Answer 1

That Wiki page is abusing language by referring to this number as a probability. You are correct that it is not. It is actually a probability per foot. Specifically, the value of 1.5789 (for a height of 6 feet) implies that the probability of a height between, say, 5.99 and 6.01 feet is close to 1.5789 [1/foot] * (6.01 - 5.99) [feet] = 0.0316 [unitless]. This value must not exceed 1, as you know. (The small range of heights (0.02 in this example) is a crucial part of the probability apparatus. It is the "differential" of height, which I will abbreviate $d(\text{height})$.) Probabilities per unit of something are called densities by analogy to other densities, like mass per unit volume.

Bona fide probability densities can have arbitrarily large values, even infinite ones.

This example shows the probability density function for a Gamma distribution (with shape parameter of $3/2$ and scale of $1/5$). Because most of the probability is less than $1$, the curve has to rise higher than $1$ in order to have a total area of $1$ as required for all probability distributions.

This density (for a beta distribution with parameters $1/2, 1/10$) becomes infinite at $0$ and at $1$. The total area still is finite (and equals $1$)!

---

The value of 1.5789 /foot is obtained in that example by estimating that the heights of males have a normal distribution with mean 5.855 feet and variance 3.50e-2 square feet. (This can be found in a previous table.) The square root of that variance is the standard deviation, 0.18717 feet. We re-express 6 feet as the number of SDs from the mean:

$$z = (6 - 5.855) / 0.18717 = 0.7747$$

The division by the standard deviation produces a relation

$$dz = d(\text{height})/0.18717$$

The Normal probability density, by definition, equals

$$\frac{1}{\sqrt{2 \pi}}\exp(-z^2/2)dz = 0.29544\ d(\text{height}) / 0.18717 = 1.5789\ d(\text{height}).$$

(Actually, I cheated: I simply asked Excel to compute NORMDIST(6, 5.855, 0.18717, FALSE). But then I really did check it against the formula, just to be sure.) When we strip the essential differential $d(\text{height})$ from the formula only the number $1.5789$ remains, like the Cheshire Cat's smile. We, the readers, need to understand that the number has to be multiplied by a small difference in heights in order to produce a probability.

Answer 2

This is a common mistake from not understanding the difference between probability mass functions, where the variable is discrete, and probability density functions, where the variable is continuous. See What is a probability distribution:

continuous probability functions are defined for an infinite number of points over a continuous interval, the probability at a single point is always zero. Probabilities are measured over intervals, not single points. That is, the area under the curve between two distinct points defines the probability for that interval. This means that the height of the probability function can in fact be greater than one. The property that the integral must equal one is equivalent to the property for discrete distributions that the sum of all the probabilities must equal one.

Answer 3

The point value at a particular parameter value of a probability density plot would be a likelihood, right? If so, then the statement might be corrected by simply changing P(height|male) to L(height|male).