7
$\begingroup$

Is there an easy way or simple example to get why it must be that

$$ p(A) \times p(B|A) = p(B) \times p(A|B) $$?

I get that $p(A) \times p(B|A)$ can be seen as probability of $B$ occurring, weighted by the probability of $A$ occurring on which it is being conditioned on. What's the background for this equation and is it generally valid?


Edited Question

Is there an easy way or simple example to get why then it must be that

$$ p(A \cap B) = p(A) p(B|A) $$?

If I think of $A$ and $B$ as overlapping circles in a Venn-Diagramm, then it seems to me that $B|A$ already would describe the overlap.

$\endgroup$
5
  • 4
    $\begingroup$ It holds because $A\cap B=B\cap A$. Remember the definition of conditional probability: $P(A\mid B)=P(A\cap B)/P(A)$, if $P(A)>0$. $\endgroup$
    – Zen
    Feb 14, 2015 at 13:39
  • $\begingroup$ Unless you define what exactly you understand by $p(B\mid A)$ and $p(A\mid B)$, the question is difficult to answer. If you mean by $P(B \mid A)$ the ratio $P(B \cap A)/P(A)$ (provided that $P(A) > 0$) as pointed out to you by @Zen, and are willing to allow the interchange of $A$ and $B$ in your definition and accept that $P(A\mid B)$ is, by definition, the ratio $P(A \cap B)/P(B)$ provided that $P(B) > 0$, then Zen's comment that $A\cap B = B \cap A$ contains all that you need to arrive at the result. $\endgroup$ Feb 14, 2015 at 16:33
  • $\begingroup$ Are you looking for a "philosophical" answer to what we mean by the notation "P(A|B)" ? $\endgroup$ Feb 14, 2015 at 17:03
  • $\begingroup$ I mean "What's the background for this equation and is it generally valid?" is pretty broad. $\endgroup$ Feb 14, 2015 at 17:10
  • $\begingroup$ I guess my problem is with the definition. Where does it come from? But that's usually the point with definitions. $\endgroup$
    – TMOTTM
    Feb 14, 2015 at 17:51

7 Answers 7

7
$\begingroup$

Short answer in words: they're both equal to the probability of (A and B) occurring.

(Probability of A) times (Probability of B, given that A has happened) equals (Probability that both A and B happen).

Similarly, (Probability of B) times (Probability of A, given that B has happened) equals (Probability that both A and B happen).

$\endgroup$
1
  • $\begingroup$ That's also a good way of putting it. $\endgroup$
    – TMOTTM
    Feb 14, 2015 at 20:13
7
$\begingroup$

First, recall

$$P(A|B)=\frac{P(A∩B)}{P(B)}$$

and consequently

$$P(A∩B)=P(A|B)P(B)$$

The trick here is to realize the very simple fact that $P(A∩B)= P(B∩A)$. This fact is quite intuitive; the probability that UNC wins and Duke loses is the same as the probability that Duke loses and UNC wins. So, in reality, we have two options:

$$P(A∩B)=P(A|B)P(B)$$

and

$$P(B∩A)=P(B|A)P(A)$$

and so

$$P(B|A)P(A)=P(A|B)P(B)$$

$\endgroup$
10
  • 1
    $\begingroup$ Why is is that so many statisticians (or maybe just wannabe statisticians in view of the soubriquet TrynnaDoStat) insist that Bayes' rule states that $$P(A\mid B)=\frac{P(A\cap B)}{P(B)}?$$ Isn't this just the definition of the conditional probability of $A$ given that $B$ occurred (provided that $P(B) > 0$? Bayes' formula or Bayes' theorem deals with what the Reverend called inverse probability: it tells you what $P(B\mid A)$ is in terms of $P(A\mid B)$. $\endgroup$ Feb 14, 2015 at 15:51
  • $\begingroup$ I don't want to get into a discussion about names, it could be called Chicken Dance Theorem for all I care, but en.wikipedia.org/wiki/Bayes%27_theorem#Statement_of_theorem $\endgroup$ Feb 14, 2015 at 16:03
  • $\begingroup$ @TrynnaDoStat You're actually taking my problem from the end. To clarify, my question is much rather, how come that $p(A | B) p(B) = p(A \cap B)$? Rearranging this to whatever other form is not the problem. But I think I understand better now that I have the hint about the combination of A and B. $\endgroup$
    – TMOTTM
    Feb 14, 2015 at 16:04
  • 1
    $\begingroup$ @TrynnadDoStat Have you even bothered to read the link that you have referred me to? It does not say that Bayes's rule or Bayes' theorem (or the Chicken Dance Theorem if you will) is what you claim it is. -1 pending an edit of your answer. $\endgroup$ Feb 14, 2015 at 16:08
  • 1
    $\begingroup$ @TrynnaDoStat OK, I reversed my downvote, but you really should take out another reference to Bayes' theorem too. You are correct in asking TMOTTM to revise his question. $\endgroup$ Feb 14, 2015 at 16:41
7
$\begingroup$

If $A$ and $B$ are events such that $P(A), P(B) > 0$, how can I compute the probability that both events occurred? That is, how can I determine $P(A\cap B)$? Well, if $A$ and $B$ are independent events, then I know that $$P(A\cap B) = P(A)P(B)\tag{1}$$ (that is just the definition of independence) and so $P(A\cap B)$ is straightforward to compute. Wouldn't it be wonderful if I could write $P(A\cap B)$ in general as $P(A)\alpha(B;A)$ where $\alpha(B;A)$ is some quantity (obviously dependent on $A$ and $B$) with the magical property that that $P(A)\alpha(B;A)$ has value $P(A \cap B)$? Now for all this to happen, we must have that $\alpha(B;A) = P(B)$ when $A$ and $B$ are independent (so that $(1)$ holds). We can say a bit more about this magical function. Since $(A\cap B) \subset A$, and so $P(A\cap B) \leq P(A)$, $\alpha(B;A)$ has maximum value $1$, and since it is possible that $P(A\cap B) = 0$, $\alpha(B;A)$ can be as small as $0$. So, since $\alpha(B;A)$ always has value in $[0,1]$, that is, it looks like a probability and quacks (acts?) like a probability, we could even call it a probability if we like, except we have not really said what $\alpha(B;A)$ is the probability of. But whatever this thingy is or what it means, we always have that $$\begin{align} P(A\cap B) &= P(A)\alpha(B;A) \tag{2}\\ \alpha(B;A) &= \frac{P(A\cap B)}{P(A)}\tag{3} \end{align}$$

Similarly, if we interchange the roles of $A$ and $B$ in the above, we can write $$\begin{align} P(B\cap A) &= P(B)\alpha(A;B) \tag{4}\\ \alpha(A;B) &= \frac{P(B\cap A)}{P(B)}\tag{5} \end{align}$$ and since $A\cap B = B \cap A$, we can use $(2)$ and $(4)$ to deduce that $$P(A)\alpha(B;A) = P(B)\alpha(A;B)\tag{6}$$ which looks almost the same as what the OP is asking about.

So, what are these quantities $\alpha(B;A)$ and $\alpha(A;B)$ that are "defined" by $(3)$ and $(5)$? Well, one way to think about this is to consider that in order for both $A$ and $B$ to have occurred, obviously $A$ must have occurred (which has probability $P(A)$), and given that we have already assumed that $A$ has occurred, we should think of $\alpha(B;A)$ as the conditional probability of $B$ given that we have already assumed that $A$ has occurred. Thus, we call $\alpha(B;A)$ as the

conditional probability of $B$ given that $A$ has occurred, denoted $P(B\mid A)$ and defined as $$ P(B\mid A) = \frac{P(A\cap B)}{P(A)} ~ \text{provided that} ~ P(A) > 0.\tag{7}$$

Notice that when $A$ and $B$ are independent, $P(B\mid A) = P(B)$, that is, knowing that $A$ has occurred does not in the least change your estimate of the probability of $B$.

All of which is fine and dandy, but what is the intuition behind all this? Well, many people make sense of probabilities as long-term relative frequencies and so let us consider a sequence of $N$ trials of the experiment, $N$ large. If the event $A$ occurred $N_A$ times on these $N$ trials and the event $B$ occurred on $N_B$ trials, then $$P(A) \approx \frac{N_A}{N}, \quad P(B) \approx \frac{N_B}{N}.$$ Similarly, $$P(A\cap B) \approx \frac{N_{A\cap B}}{N}$$ where we note that the trials on which $A\cap B$ occurred are necessarily a subset of the $N_A$ trials on which $A$ occurred (as well as a subset of the $N_B$ trials on which $B$ occurred). But, $P(B\mid A)$ is the (conditional) probability of $B$ given that event $A$ has occurred, and so let us consider the $N_A$ trials on which $A$ has occurred. On this subsequence of $N_A$ trials, what is the relative frequency of $B$? Well, $B$ occurred on exactly $N_{A\cap B}$ of these $N_A$ trials, and so $$P(B\mid A) \approx \frac{N_{A\cap B}}{N_A} = \frac{\frac{N_{A\cap B}}{N}}{\frac{N_{A}}{N}}\approx \frac{P(A\cap B)}{P(A)}$$ which matches the definition in $(7)$.


OK, so now let the down-voting begin....

$\endgroup$
3
$\begingroup$

Just trying a graphical intuition... Hope it flies...enter image description here

$\endgroup$
2
$\begingroup$

$p(B\mid A)p(A) = p(B\cap A)=p(A\mid B)p(B)$

$\endgroup$
4
  • $\begingroup$ Can you expand this a little? I'm not sure this will clarify the issue sufficiently for the OP. $\endgroup$ Feb 14, 2015 at 14:02
  • 4
    $\begingroup$ +1 I like this because it goes to the point and responds to the request for "easy way or simple example." $\endgroup$
    – whuber
    Feb 14, 2015 at 15:08
  • $\begingroup$ Just to be sure, the comma in the notation $p(A, B)$ is another way of writing $\cap$? $\endgroup$
    – TMOTTM
    Feb 14, 2015 at 16:01
  • $\begingroup$ @TMOTTM Yes, the comma and the cap both mean (A and B). $\endgroup$
    – Adrian
    Feb 14, 2015 at 16:55
1
$\begingroup$

A simple example is like this. Suppose we have A containing two events, "A = 1" or "A = 2", and B containing three events, "B = 1", "B = 2", or "B = 3". The (arbitrary) cell counts for the occurrence of the events are listed in the following table.

enter image description here

Now we can do a simple example: \begin{align*} P(A=1) & = \frac{6}{21} \\ P(B=2) &= \frac{7}{21} \\ P(A=1 \mid B=2) &= \frac{2}{7} \\ P(B=2 \mid A=1) &= \frac{2}{6} \\ \end{align*} which yields to \begin{align*} P(A=1) \times P(B=2 \mid A=1) &= \frac{6}{21} \times \frac{2}{6} = \frac{2}{21} \\ P(B=2) \times P(A=1 \mid B=2) &= \frac{7}{21} \times \frac{2}{7} = \frac{2}{21} \end{align*}

It's easy to verify that the equation does hold for other occurrence of event A and B.

The equation is generally valid. That's no magic of course.

If A and B are independent, then $P(B|A) = P(B)$, $P(A|B) = P(A)$. The two sides of the equation reduce to $P(A)P(B)$. The equation holds.

If A and B are not independent, then it's easy to be proved by the definition of conditional probability (or Bayes' theorem).

$\endgroup$
1
$\begingroup$

Let us motivate the formula for conditional probability. We keep things are simple as possible. $S$, our sample space, will be finite. Let $E$ and $F$ be subsets of it (events).

When we write, $P(E|F)$ we are saying, "probability of $E$ given $F$". Thus, instead of the possibilities being in $S$, they are now all in $F$ since we are told that event $F$ has occurred. The new sample space is $F$. To find the probability of $E$ given $F$, it remains to count the number of possibilities which are in $E$. It would be wrong to write $|E|$ since this is counting the possibilities outside of $F$ (the new sample space). Therefore, there are $|E\cap F|$ possibilities of $E$ which happen within $F$.

It follows by the finite probability formula (event size divided by sample size), $$ P(E|F) = \frac{|E\cap F|}{|F|} $$

But let us rewrite this formula to have probabilities on the right side. We use, $$ P(E|F) = \frac{|E\cap F|}{|S|} \cdot \frac{|S|}{|F|} = \left( \frac{|E\cap F|}{|S|} \right) \bigg/\left( \frac{|F|}{|S|} \right) = \frac{P(E\cap F)}{P(F)} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.