Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I suppose that
$$P(A|B) = P(A | B,C) * P(C) + P(A|B,\neg C) * P(\neg C)$$
is correct, whereas
$$P(A|B) = P(A | B,C) + P(A|B,\neg C) $$
is incorrect.
However, I have got an "intuition" about the later one, that is, you consider the probability P(A | B) by splitting two cases (C or Not C). Why this intuition is wrong?
Suppose, as an easy counter example, that the probability $P(A)$ of $A$ is $1$, regardless of the value of $C$. Then, if we take the incorrect equation, we get:
$P(A | B) = P(A | B, C) + P(A | B, \neg C) = 1 + 1 = 2$
That obviously can't be correct, a probably cannot be greater than $1$. This helps to build the intuition that you should assign a weight to each of the two cases proportional to how likely that case is, which results in the first (correct) equation..
That brings you closer to your first equation, but the weights are not completely right. See A. Rex' comment for the correct weights.
Dennis's answer has a great counter-example, disproving the wrong equation. This answer seeks to explain why the following equation is right:
$$P(A|B) = P(A|C,B) P(C|B) + P(A|\neg C,B) P(\neg C|B).$$
As every term is conditioned on $B$, we can replace the entire probability space by $B$ and drop the $B$ term. This gives us:
$$P(A) = P(A|C) P(C) + P(A|\neg C) P(\neg C).$$
Then you are asking why this equation has the $P(C)$ and $P(\neg C)$ terms in it.
The reason is that $P(A|C) P(C)$ is the portion of $A$ in $C$ and $P(A|\neg C) P(\neg C)$ is the portion of $A$ in $\neg C$ and the two add up to $A$. See diagram. On the other hand $P(A|C)$ is the proportion of $C$ containing $A$ and $P(A|\neg C)$ is the proportion of $\neg C$ containing $A$ - these are proportions of different regions so they don't have common denominators so adding them is meaningless.
I know you've already received two great answers to your question, but I just wanted to point out how you can turn the idea behind your intuition into the correct equation.
First, remember that $P(X \mid Y) = \dfrac{P(X \cap Y)}{P(Y)}$ and equivalently $P(X \cap Y) = P(X \mid Y)P(Y)$.
To avoid making mistakes, we will use the first equation in the previous paragraph to eliminate all conditional probabilities, then keep rewriting expressions involving intersections and unions of events, then use the second equation in the previous paragraph to re-introduce the conditionals at the end. Thus, we start with: $$P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}$$
We will keep rewriting the right-hand side until we get the desired equation.
The casework in your intuition expands the event $A$ into $(A \cap C) \cup (A \cap \neg C)$, resulting in $$P(A \mid B) = \dfrac{P(((A \cap C) \cup (A \cap \neg C)) \cap B)}{P(B)}$$
As with sets, the intersection distributes over the union: $$P(A \mid B) = \dfrac{P((A \cap B \cap C) \cup (A \cap B \cap \neg C))}{P(B)}$$
Since the two events being unioned in the numerator are mutually exclusive (since $C$ and $\neg C$ cannot both happen), we can use the sum rule: $$P(A \mid B) = \dfrac{P(A \cap B \cap C)}{P(B)} + \dfrac{P(A \cap B \cap \neg C)}{P(B)}$$
We now see that $P(A \mid B) = P(A \cap C \mid B) + P(A \cap \neg C \mid B)$; thus, you can use the sum rule on the event on the event of interest (the "left" side of the conditional bar) if you keep the given event (the "right" side) the same. This can be used as a general rule for other equality proofs as well.
We re-introduce the desired conditionals using the second equation in the second paragraph: $$P(A \cap (B \cap C)) = P(A \mid B \cap C)P(B \cap C)$$ and similarly for $\neg C$.
We plug this into our equation for $P(A \mid B)$ as: $$P(A \mid B) = \dfrac{P(A \mid B \cap C)P(B \cap C)}{P(B)} + \dfrac{P(A \mid B \cap \neg C)P(B \cap \neg C)}{P(B)}$$
Noting that $\dfrac{P(B \cap C)}{P(B)} = P(C \mid B)$ (and similarly for $\neg C$), we finally get $$P(A \mid B) = P(A \mid B \cap C)P(C \mid B) + P(A \mid B \cap \neg C)P(\neg C \mid B)$$
Which is the correct equation (albeit with slightly different notation), including the fix A. Rex pointed out.
Note that $P(A \cap C \mid B)$ turned into $P(A \mid B \cap C)P(C \mid B)$. This mirrors the equation $P(A \cap C) = P(A \mid C)P(C)$ by adding the $B$ condition to not only $P(A \cap C)$ and $P(A \mid C)$, but also $P(C)$ as well. I think if you are to use familiar rules on conditioned probabilities, you need to add the condition to all probabilities in the rule. And if there's any doubt whether that idea works for a particular situation, you can always expand out the conditionals to check, as I did for this answer.
Probabilities are ratios; the probability of A given B is how often A happens within the space of B. For instance, $P(\text{rain|March})$ is the number of rainy days in March divided by the number of total days in March. When dealing with fractions, it makes sense to split up numerators. For instance,
\begin{align} P(\text{rain or snow|March}) &= \frac{(\text{number of rainy or snowy days in March})}{(\text{total number of days in March})} \\[7pt] &= \frac{(\text{number of rainy days in March})}{(\text{total number of days in March)}} + \\[4pt] &\qquad \frac{\text{(number of snowy days in March)}}{(\text{total number of days in March)}} \\[7pt] &= P(\text{rain|March})+P(\text{snow|March}) \end{align}
This of course assumes that "snow" and "rain" are mutually exclusive. It does not, however, make sense to split up denominators. So if you have $P(\text{rain|February or March})$, that is equal to
$$\frac{(\text{number of rainy days in February and March})}{(\text{total number of days in February and March})}.$$
But that is not equal to
$$\frac{(\text{number of rainy days in February})}{(\text{total number of days in February})} + \frac{(\text{number of rainy days in March)}}{(\text{total number of days in March)}}.$$
If you're having trouble seeing that, you can try out some numbers. Suppose there are 10 rainy days in February and 8 in March. Then we have
$$\frac{(\text{number of rainy days in February and March})}{(\text{total number of days in February and March)}} = (10+8)/(28+31) = 29.5\% $$
and
\begin{align} \frac{(\text{number of rainy days in February})}{(\text{total number of days in February)}} + \frac{(\text{number of rainy days in March)}}{(\text{total number of days in March)}} &= (10/28)+(8/31) \\ &= 35.7\% + 25.8\% \\ &= 61.5\% \end{align}
The first number, 29.5%, is the average of 35.7% and 25.8% (with the second number weighted slightly more because there is are more days in March). When you say $P(A|B)=P(A|B,C)+P(A|B,¬C)$ you're saying that $\frac{x_1+x_2}{y_1+y_2} = \frac{x_1}{y_1}+\frac{x_2}{y_2}$, which is false.
If I go to Spain, I can get sunburnt. $$P(sunburnt | Spain)=0.2$$ This tells me nothing about getting sunburnt if not going to Spain, let's say $$P(sunburnt|\neg Spain)=0.1$$ This year I'm going to Spain, so $$P(sunburnt)=0.2$$ Letting $B=\Omega$, this is, $P(B)=1$, your intuition would imply $$P(A)=P(A|C)+P(A|\neg C)$$ which by the previous argument, isn't neccesarily true.
