15
$\begingroup$

I suppose that

$$P(A|B) = P(A | B,C) * P(C) + P(A|B,\neg C) * P(\neg C)$$

is correct, whereas

$$P(A|B) = P(A | B,C) + P(A|B,\neg C) $$

is incorrect.

However, I have got an "intuition" about the later one, that is, you consider the probability P(A | B) by splitting two cases (C or Not C). Why this intuition is wrong?

$\endgroup$
  • 4
    $\begingroup$ Here's a simple example to test out your equations. Toss two independent, fair coins. Let $A$ be the event that the first comes up heads, $B$ be the event that the second comes up heads, and $C$ be the event that both come up heads. Is either equation you wrote correct? $\endgroup$ – A. Rex Mar 19 '18 at 13:38
  • 4
    $\begingroup$ The law of total probability says if you want to express an unconditional probability as a sum of conditional probabilities, you must weight by the conditioning event: e.g. $P(A) = P(A|B)P(B) + P(A|\bar{B})P(\bar{B})$ $\endgroup$ – AdamO Mar 19 '18 at 14:55
25
$\begingroup$

Suppose, as an easy counter example, that the probability $P(A)$ of $A$ is $1$, regardless of the value of $C$. Then, if we take the incorrect equation, we get:

$P(A | B) = P(A | B, C) + P(A | B, \neg C) = 1 + 1 = 2$

That obviously can't be correct, a probably cannot be greater than $1$. This helps to build the intuition that you should assign a weight to each of the two cases proportional to how likely that case is, which results in the first (correct) equation..


That brings you closer to your first equation, but the weights are not completely right. See A. Rex' comment for the correct weights.

$\endgroup$
  • 1
    $\begingroup$ Should the weights in the "first (correct) equation" be $P(C)$ and $P(\neg C)$, or should they be $P(C\mid B)$ and $P(\neg C\mid B)$? $\endgroup$ – A. Rex Mar 19 '18 at 13:19
  • $\begingroup$ @A.Rex That's a good point, for full correctness I think it should be $P(C | B)$ and $P(\neg C | B)$. Everything (just a single term) on the left-hand side of the equation assumes that $B$ is given, so without any additional assumptions (like assuming that $B$ and $C$ are independent of each other), the same should be the case on the right-hand side $\endgroup$ – Dennis Soemers Mar 19 '18 at 13:36
  • $\begingroup$ Just think of A|B being 200% sure to happen. $\endgroup$ – Mark L. Stone Mar 19 '18 at 14:57
  • $\begingroup$ @MarkL.Stone Does that mean it always happens twice? ;) $\endgroup$ – Solomonoff's Secret Mar 19 '18 at 20:22
9
$\begingroup$

Dennis's answer has a great counter-example, disproving the wrong equation. This answer seeks to explain why the following equation is right:

$$P(A|B) = P(A|C,B) P(C|B) + P(A|\neg C,B) P(\neg C|B).$$

As every term is conditioned on $B$, we can replace the entire probability space by $B$ and drop the $B$ term. This gives us:

$$P(A) = P(A|C) P(C) + P(A|\neg C) P(\neg C).$$

Then you are asking why this equation has the $P(C)$ and $P(\neg C)$ terms in it.

The reason is that $P(A|C) P(C)$ is the portion of $A$ in $C$ and $P(A|\neg C) P(\neg C)$ is the portion of $A$ in $\neg C$ and the two add up to $A$. See diagram. On the other hand $P(A|C)$ is the proportion of $C$ containing $A$ and $P(A|\neg C)$ is the proportion of $\neg C$ containing $A$ - these are proportions of different regions so they don't have common denominators so adding them is meaningless.

pic

$\endgroup$
  • 2
    $\begingroup$ Not "everything is conditioned on $B$". In particular, $P(C)$ and $P(\neg C)$ are not, so you can't just drop $B$. Moreover, this might suggest the equation is wrong! $\endgroup$ – A. Rex Mar 19 '18 at 13:12
  • $\begingroup$ @A.Rex Technically you're right, I should have said every term involving $A$ is conditioned on $B$ (I made a simple substitution $A|B \rightarrow A$). I will correct the answer. $\endgroup$ – Solomonoff's Secret Mar 19 '18 at 13:18
  • 5
    $\begingroup$ My objection wasn't a technicality. Your diagram correctly proves that $P(A) = P(A\mid C) P(C) + P(A\mid\neg C) P(\neg C)$, which after conditioning on $B$ becomes $P(A\mid B) = P(A\mid B,C) P(C\mid B) + P(A\mid B,\neg C) P(\neg C\mid B)$; note that the probabilities of $C$ and $\neg C$ are also conditioned on $B$. This is not the first equation given in the OP, which is good news, because the first equation given in the OP is not correct. $\endgroup$ – A. Rex Mar 19 '18 at 13:31
  • $\begingroup$ @A.Rex You are right once again, $C$ must also conditioned on $B$ as the proportion of the probability space contained in $C$ might not be the same as the proportion of $B$ contained in $C$. This point escaped me. I will revise again. $\endgroup$ – Solomonoff's Secret Mar 19 '18 at 13:33
7
$\begingroup$

I know you've already received two great answers to your question, but I just wanted to point out how you can turn the idea behind your intuition into the correct equation.

First, remember that $P(X \mid Y) = \dfrac{P(X \cap Y)}{P(Y)}$ and equivalently $P(X \cap Y) = P(X \mid Y)P(Y)$.

To avoid making mistakes, we will use the first equation in the previous paragraph to eliminate all conditional probabilities, then keep rewriting expressions involving intersections and unions of events, then use the second equation in the previous paragraph to re-introduce the conditionals at the end. Thus, we start with: $$P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}$$

We will keep rewriting the right-hand side until we get the desired equation.

The casework in your intuition expands the event $A$ into $(A \cap C) \cup (A \cap \neg C)$, resulting in $$P(A \mid B) = \dfrac{P(((A \cap C) \cup (A \cap \neg C)) \cap B)}{P(B)}$$

As with sets, the intersection distributes over the union: $$P(A \mid B) = \dfrac{P((A \cap B \cap C) \cup (A \cap B \cap \neg C))}{P(B)}$$

Since the two events being unioned in the numerator are mutually exclusive (since $C$ and $\neg C$ cannot both happen), we can use the sum rule: $$P(A \mid B) = \dfrac{P(A \cap B \cap C)}{P(B)} + \dfrac{P(A \cap B \cap \neg C)}{P(B)}$$

We now see that $P(A \mid B) = P(A \cap C \mid B) + P(A \cap \neg C \mid B)$; thus, you can use the sum rule on the event on the event of interest (the "left" side of the conditional bar) if you keep the given event (the "right" side) the same. This can be used as a general rule for other equality proofs as well.

We re-introduce the desired conditionals using the second equation in the second paragraph: $$P(A \cap (B \cap C)) = P(A \mid B \cap C)P(B \cap C)$$ and similarly for $\neg C$.

We plug this into our equation for $P(A \mid B)$ as: $$P(A \mid B) = \dfrac{P(A \mid B \cap C)P(B \cap C)}{P(B)} + \dfrac{P(A \mid B \cap \neg C)P(B \cap \neg C)}{P(B)}$$

Noting that $\dfrac{P(B \cap C)}{P(B)} = P(C \mid B)$ (and similarly for $\neg C$), we finally get $$P(A \mid B) = P(A \mid B \cap C)P(C \mid B) + P(A \mid B \cap \neg C)P(\neg C \mid B)$$

Which is the correct equation (albeit with slightly different notation), including the fix A. Rex pointed out.

Note that $P(A \cap C \mid B)$ turned into $P(A \mid B \cap C)P(C \mid B)$. This mirrors the equation $P(A \cap C) = P(A \mid C)P(C)$ by adding the $B$ condition to not only $P(A \cap C)$ and $P(A \mid C)$, but also $P(C)$ as well. I think if you are to use familiar rules on conditioned probabilities, you need to add the condition to all probabilities in the rule. And if there's any doubt whether that idea works for a particular situation, you can always expand out the conditionals to check, as I did for this answer.

$\endgroup$
  • 2
    $\begingroup$ +1. I think you extracted the equation that OP tried to intuit: $P(A\mid B)=P(A\cap C\mid B)+P(A\cap\neg C\mid B)$. $\endgroup$ – A. Rex Mar 19 '18 at 16:00
  • $\begingroup$ Thanks! That was the main point I wanted to make, but couldn't figure out a high-level explanation why the intersection goes on the left rather than the right, so I used formulas instead. Also, I just noticed you were the one who pointed out the mistake in OP's formula, so I credited you for that. (I probably wouldn't have noticed either, lol.) $\endgroup$ – YawarRaza7349 Mar 19 '18 at 16:25
2
$\begingroup$

Probabilities are ratios; the probability of A given B is how often A happens within the space of B. For instance, $P(\text{rain|March})$ is the number of rainy days in March divided by the number of total days in March. When dealing with fractions, it makes sense to split up numerators. For instance,

\begin{align} P(\text{rain or snow|March}) &= \frac{(\text{number of rainy or snowy days in March})}{(\text{total number of days in March})} \\[7pt] &= \frac{(\text{number of rainy days in March})}{(\text{total number of days in March)}} + \\[4pt] &\qquad \frac{\text{(number of snowy days in March)}}{(\text{total number of days in March)}} \\[7pt] &= P(\text{rain|March})+P(\text{snow|March}) \end{align}

This of course assumes that "snow" and "rain" are mutually exclusive. It does not, however, make sense to split up denominators. So if you have $P(\text{rain|February or March})$, that is equal to

$$\frac{(\text{number of rainy days in February and March})}{(\text{total number of days in February and March})}.$$

But that is not equal to

$$\frac{(\text{number of rainy days in February})}{(\text{total number of days in February})} + \frac{(\text{number of rainy days in March)}}{(\text{total number of days in March)}}.$$

If you're having trouble seeing that, you can try out some numbers. Suppose there are 10 rainy days in February and 8 in March. Then we have

$$\frac{(\text{number of rainy days in February and March})}{(\text{total number of days in February and March)}} = (10+8)/(28+31) = 29.5\% $$

and

\begin{align} \frac{(\text{number of rainy days in February})}{(\text{total number of days in February)}} + \frac{(\text{number of rainy days in March)}}{(\text{total number of days in March)}} &= (10/28)+(8/31) \\ &= 35.7\% + 25.8\% \\ &= 61.5\% \end{align}

The first number, 29.5%, is the average of 35.7% and 25.8% (with the second number weighted slightly more because there is are more days in March). When you say $P(A|B)=P(A|B,C)+P(A|B,¬C)$ you're saying that $\frac{x_1+x_2}{y_1+y_2} = \frac{x_1}{y_1}+\frac{x_2}{y_2}$, which is false.

$\endgroup$
1
$\begingroup$

If I go to Spain, I can get sunburnt. $$P(sunburnt | Spain)=0.2$$ This tells me nothing about getting sunburnt if not going to Spain, let's say $$P(sunburnt|\neg Spain)=0.1$$ This year I'm going to Spain, so $$P(sunburnt)=0.2$$ Letting $B=\Omega$, this is, $P(B)=1$, your intuition would imply $$P(A)=P(A|C)+P(A|\neg C)$$ which by the previous argument, isn't neccesarily true.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.