# Naive Bayes Derivation

I was going over the derivation of Naive Bayes, and the following 3 lines were given:

Suppose $X = \left < X_1, X_2 \right>$

\begin{align} P(X|Y) &= P(X_1, X_2 | Y) \\[2pt] &= P(X_1 | X_2, Y)P(X_2 | Y) \\[2pt] &= P(X_1 | Y)P(X_2 | Y) \end{align}

So the third line comes from the fact that we have made the assumption that each $X_i$ is conditionally independent given Y, but how was line 2 derived?

• Would the second line be obvious if you were to erase all instances of "$Y$" from the notation?
– whuber
May 31, 2016 at 1:51
• @whuber Not really. How is the probability of A and B given C equal to the probability of A, given B and C, times the probability of B given C? That can't be Bayes' theorem as the answer suggests. May 31, 2016 at 2:15
• If you condition everything on $Y$, then it is not necessary to reference $Y$ explicitly in the notation. The formula reduces to the most basic property of conditional probability.
– whuber
May 31, 2016 at 2:18
• Oh, so does $P(X_1, X_2)$ mean $P(X_1 and X_2)$? If so, then that's easy! May 31, 2016 at 2:21
• I would understand it to mean the joint distribution of $(X_1,X_2)$.
– whuber
May 31, 2016 at 2:23

I've figured it out.

The second line comes from the "chain rule" of probability which states that:

$$P(A, B, \dots, Z) = P(A |B,\dots,Z) \times P(B|C,\dots,Z) \times \dots \times P(Y|Z) \times P(Z)$$

In fact, this can be proven by expanding each term in the equality:

$$P(A, B, ..., Z) = \frac{P(A |B,\dots,Z)}{P(B,\dots,Z)} \times \frac{P(B |C,\dots,Z)}{P(C,\dots,Z)} \times \dots \times \frac{P(Y|Z)}{P(Z)} \times P(Z)$$

and we can see that the terms cancel out.

My main confusion stemmed from the fact that I didn't know that the commas were in fact a notation for the intersection. This post clarifies it:

Now, by common convention, a list of events is interpreted as the intersection (AND) of those events, such that $P(E_1,E_2) = P(E_1 \cap E_2)$ or, using logical connectives instead of set-theory ones, $P(E_1 \land E_2)$. However, that convention is by no means universal, so if you want to be sure to avoid ambiguity, you should explicitly use $\cap$ (or $\land$) to denote the intersection of events.