30
$\begingroup$

The formula for the conditional probability of $\text{A}$ happening given that $\text{B}$ has happened is:$$ P\left(\text{A}~\middle|~\text{B}\right)=\frac{P\left(\text{A} \cap \text{B}\right)}{P\left(\text{B}\right)}. $$

My textbook explains the intuition behind this in terms of a Venn diagram.

enter image description here

Given that $\text{B}$ has occurred, the only way for $\text{A}$ to occur is for the event to fall in the intersection of $\text{A}$ and $\text{B}$.

In that case, wouldn't the probability of $P\left(\text{A} \middle| \text{B}\right)$ simply be equal to the probability of $\text{A}$ intersection $\text{B}$, since that's the only way the event could happen? What am I missing?

$\endgroup$
  • 7
    $\begingroup$ Do you have an intuitive understanding of what conditional probability "is", if we forget for a while how to compute it? $\endgroup$ – Juho Kokkala Feb 1 '18 at 16:37
  • 4
    $\begingroup$ By conditioning on B (the event that has occured), you restrict your space of outcomes from $\Omega$ (the whole plane) to B only. You forget everything that is outside B. The probability of event A has to be measured with respect B, since the probability is between 0 and 1. $\endgroup$ – Vladislavs Dovgalecs Feb 1 '18 at 17:59
  • 1
    $\begingroup$ You're missing the fact that the white part of the Event A circle is no longer part of the population once you know Event B occurred. $\endgroup$ – Monty Harder Feb 1 '18 at 21:21
  • 4
    $\begingroup$ Intuitions are not exact, nor are they singular, so why ask about the (singular) exact intuition? A useful intuition suffices, but not all suggestions will be useful to all people. $\endgroup$ – John Coleman Feb 2 '18 at 14:25

12 Answers 12

23
$\begingroup$

A good intuition is given that B occurred—with or without A—what is the probability of A? I.e., we are now in the universe in which B occurred—the full right circle. In that circle, the probability of A is the area of A intersect B divided by the area of the circle.

$\endgroup$
  • 5
    $\begingroup$ In other words -- I tell you $B$ happened, which means we live in the $B$ circle. Within that world, what % of events are in the lens ($A\cap B$)? $\endgroup$ – MichaelChirico Feb 3 '18 at 12:30
17
$\begingroup$

I would think of it like this: I take for granted that you understand the intuition until:

Given that B has occurred, the only way for A to occur is for the even to fall in the intersection of A & B.

and I am going to comment the second image you posted:

  1. Imagine that the entire white rectangle is your sample space $\Omega$.

    Assigning a probability to a set means that you are measuring in some sense that set. It is the same as if you measured the area of the rectangle but the probability is a different kind of measure that has specific properties (I won't say anything more about this).

  2. You know that $P(\Omega) = 1$ and this is interpreted like this:

    $\Omega$ represents all the events that could happen and something has to happen so we have 100% probability that something happens.

  3. Analogously the set $A$ has a probability $P(A)$ that is proportional to the probability of the sample space $\Omega$. Graphically speaking you see that $A \subset \Omega$ hence the measure of $A$ (its probability $P(A)$) has to be lesser than $P(\Omega)$. The same reasoning is valid for the set $A \cap B$. This set can be measured and its measure is $P(A \cap B)$.

  4. If now you are told that $B$ has happened you have to think as if $B$ were your "new" $\Omega$. If $B$ is your "new" $\Omega$ then you can be 100% sure that everything happens in the set $B$.

    And what does that mean? It means that now, in the "new" contest $P(B \mid B) =1$, and you have to rescale all the probability measures, taking into account that they have to be expressed in terms of the "new" sample space $B$. It is a simple proportion.

    Your intuition is almost right when you say that:

the probability of P(A | B) would simply be equal to the probability of A intersection B

and the "almost" is due to the fact that now your sample space has changed (it is $B$ now) and you want to rescale $P(A \cap B)$ accordingly.

  1. $P(A \mid B)$ is your $P(A \cap B)$ in the new world where the sample space is now $B$. In words you would say it like this (and please try to visualize it on the image with the sets):

    In the new world the ratio between the measure of $B$ and the measure of $A\cap B$ must be the the same as the ratio between the measure of $\Omega$ and the measure of $A \mid B$

  2. Lastly translate this in mathematical language (a simple proportion):

$$P(B):P(A \cap B) = P(\Omega):P(A \mid B)$$

and since $P(\Omega)=1$ it follows that:

$$P(A \mid B)= P(A \cap B):P(B)$$

$\endgroup$
5
$\begingroup$

You'll see intuition easily thinking about the following problem.

Suppose, you have 10 balls: 6 Black and 4 red. Of Black balls 3 are Awesome and of red balls only 1 is Awesome. How likely it is that a a Black ball is also Awesome?

The answer is very easy: it's 50%, because we have 3 Awesome Black balls out of total 6 Black balls.

This is how you map probabilities to our problem:

  • 3 balls that are Black AND Awesome correspond to $P(A \cap B)$
  • 6 balls that are Black correspond to $P(B)$
  • probability that a ball is Awesome when we KNOW that it is Black: $P(A\mid B)$
$\endgroup$
  • 1
    $\begingroup$ Would it not make more sense to write $n(B)=6$ rather than $P(B)=6$? $\endgroup$ – Silverfish Feb 2 '18 at 22:31
  • $\begingroup$ @Silverfish It would be more accurate, but I was after the intuition in this case $\endgroup$ – Aksakal Feb 3 '18 at 1:21
4
$\begingroup$

For a basic intuition of the conditional probability formula, I always like using a two way table. Let's say there are 150 students in a yeargroup, of whom 80 are female and 70 male, each of whom must study exactly one language course. The two-way table of students taking different courses is:

        | French   German   Italian  | Total
-------- --------------------------- -------
Male    |     30       20        20  |    70
Female  |     25       15        40  |    80
-------- --------------------------- -------
Total   |     55       35        60  |   150

Given that a student takes the Italian course, what is the probability they are female? Well the Italian course has 60 students, of whom 40 are females studying Italian, so the probability must be:

$$P(\text{F|Italian})=\frac{n(\text{F} \cap \text{Italian})}{n(\text{Italian})}=\frac{40}{60}=\frac{2}{3}$$

where $n(A)$ is the cardinality of the set $A$, i.e. the number of items it contains. Note that we needed to use $n(\text{F} \cap \text{Italian})$ in the numerator and not just $n(\text{F})$, because the latter would have included all 80 females, including the other 40 who do not study Italian.

But if the question were flipped around, what is the probability that a student takes the Italian course, given that they are female? Then 40 of the 80 female students take the Italian course, so we have:

$$P(\text{Italian|F})=\frac{n(\text{Italian} \cap \text{F})}{n(\text{F})}=\frac{40}{80}=\frac{1}{2}$$

I hope this provides intuition for why

$$P(A|B)=\frac{n(A \cap B)}{n(B)}$$

Understanding why the fraction can be written with probabilities instead of cardinalities is a matter of equivalent fractions. For example, let us return to the probability a student is female given that they are studying Italian. There are 150 students in total, so the probability that a student is female and studies Italian is 40/150 (this is a "joint" probability) and the probability a student studies Italian is 60/150 (this is a "marginal" probability). Note that dividing the joint probability by the marginal probability gives:

$$\frac{P(\text{F} \cap \text{Italian})}{P(\text{Italian})}=\frac{40/150}{60/150}=\frac{40}{60}=\frac{n(\text{F} \cap \text{Italian})}{n(\text{Italian})}=P(\text{F|Italian})$$

(To see that the fractions are equivalent, multiplying numerator and denominator by 150 removes the "/150" in each.)

More generally, if your sampling space $\Omega$ has cardinality $n(\Omega)$ — in this example the cardinality was 150 — we find that

$$P(A|B)=\frac{n(A \cap B)}{n(B)}=\frac{n(A \cap B)/n(\Omega)}{n(B)/n(\Omega)}=\frac{P(A \cap B)}{P(B)}$$

$\endgroup$
3
$\begingroup$

I would reverse the logic. The probability that both $A$ and $B$ is either:

  1. The probability $B$ happened, and that given that $A$ happened.
  2. Same but reverse roles for $A$ and $B$

This will give you

$p(A\cap B)=p(B)p(A\mid B)$

If you're looking for a negative to your suggestion, it's while it's true the probability of $A$ given $B$ is contained in the probability of the product, the space you're rolling the dice in is smaller than your original probability space - you know for sure you're "in" $B$, hence you divide by the size of the new space.

$\endgroup$
2
$\begingroup$

The Venn diagram doesn't represent probability, it represents the measure of subsets of the event space. A probability is the ratio between two measures; the probability of X is the size of "everything that constitutes X" divided the size of "all the events being considered". Any time you're calculating a probability, you need both a "success space" and a "population space". You can't calculated a probability based just on "how big" the success space is. For instance, the probability of rolling a seven with two dice is the number of ways of rolling a seven divided by the total number of ways of rolling two dice. Just knowing the number of ways of rolling a seven is not enough to calculate the probability. P(A|B) is the ratio of the measure of the "both A and B happen" space and the measure of the "B happens" space. That's what the "|" means: it means "make what comes after this the population space".

$\endgroup$
2
$\begingroup$

I think the best way to think about this is drawing step-by-step paths.

Let's describe Event B as rolling a $4$ on a fair die - this can be easily shown to have probability $\frac{1}{6}$. Now let's describe Event A as drawing an Ace from a standard 52-card deck of cards - this can be easily shown to have probability $\frac{1}{13}$.

Let's now run an experiment where we roll a die and then pick a card. So $P\left(A \middle| B\right)$ would be the probability that we draw an Ace, given that we have already rolled a $4$. If you look at the image, this would be the $\frac{1}{6}$ path (go up) and then the $\frac{1}{13}$ path (go up again).

Intuitively, the total probability space is what we have already been given: rolling the $4$. We can ignore the $\frac{1}{13}$ and $\frac{12}{13}$ the initial down path leads to, since it was GIVEN that we rolled a $4$. By law of multiplication, our total space is then $\left(\frac{1}{6}{\times}\frac{1}{13}\right) + \left(\frac{1}{6}{\times}\frac{12}{13}\right)$.

Now what's the probability we drew an Ace, GIVEN that we rolled a $4$? The answer by using the path is $\left(\frac{1}{6} {\times} \frac{1}{13} \right)$, which we then need to divide by the total space. So we get$$ P\left(A \middle| B\right) = \frac{\frac{1}{6} {\times} \frac{1}{13}}{\left(\frac{1}{6} {\times} \frac{1}{13}\right) + \left(\frac{1}{6} {\times} \frac{12}{13}\right)}. $$

enter image description here

$\endgroup$
  • 2
    $\begingroup$ I was wondering what the downvote was for, because probability trees can be very instructive. Perhaps the concern is that using independent events for the illustration misses the very point of conditional probability, which is that the probability distribution can change depending on the conditioning event. Using a less-superficial illustration may help. $\endgroup$ – whuber Feb 1 '18 at 22:52
1
$\begingroup$

Think of it on terms of counts. Marginal probability is how many times A occurred divided by sample size. Joint probability of A and B is how many times A occurred together with B divided by sample size. Conditional probability of A given B is how many times A occurred together with B divided by how many times B occurred, i.e. only the A's "within" B's.

You can find nice visual illustration on this blog, that shows it using Lego blocks.

$\endgroup$
1
$\begingroup$

At the time of writing there is about 10 answers which seem to all miss the most important point: you are essentially right.

In that case, wouldn't the probability of P(A | B) simply be equal to the probability of A intersection B, since that's the only way the event could happen?

This is definitely true. This explains why the quantity we to define $P(A\vert B)$ is actually $P(A \cap B)$ rescaled.

What am I missing?

You are missing that the probability of B being satisfied given that B is satisfied should be 1 since this is quite a certain event, and not $P(B\cap B) = P(B)$ which can well be less than 1. Dividing by $P(B)$ makes the conditional probability of B given B equal to 1, as expected. Actually this is even better and makes the map $A \mapsto P(A\vert B)$ a probability – so a conditional probability is actually a probability.

$\endgroup$
0
$\begingroup$

I feel it is more intuitive when we have a concrete data to estimate the probabilities.

Let's use mtcars data as an example, the data looks like this (we only use number of cylinders and transmission type.)

> mtcars[,c("am","cyl")]
                    am cyl
Mazda RX4            1   6
Mazda RX4 Wag        1   6
Datsun 710           1   4
Hornet 4 Drive       0   6
...  
...
Ford Pantera L       1   8
Ferrari Dino         1   6
Maserati Bora        1   8
Volvo 142E           1   4

We can calculate the joint distribution on two variables by doing a cross table:

> prop.table(table(mtcars$cyl,mtcars$am))

          0       1
  4 0.09375 0.25000
  6 0.12500 0.09375
  8 0.37500 0.06250

The joint probability means we want to consider two variables at the same time. For example, we will ask how many cars are 4 cylinder and manual transmission.

Now, we come to conditional probability. I found the most intuitive way to explain conditional probability is using the term filtering on data.

Suppose we want to get $P(am=1|cyl=4)$, we will do following estimations:

> cyl_4_cars=subset(mtcars, cyl==4)
> prop.table(table(cyl_4_cars$am))

        0         1 
0.2727273 0.7272727 

This means, we only care cars have 4 cylinder. So we filter data on that. After filtering, we check how many of them are manual transmission.

You can compare conditional this with joint I mentioned earlier to feel the differences.

$\endgroup$
0
$\begingroup$

If A were a superset of B the probability that A happens is always 1 given that B happened, i.e. P(A|B) = 1. However, B itself may have a probability much smaller than 1.

Consider the following example:

  • given x is a natural number in 1..100,
  • A is 'x is an even number'
  • B is 'x is divisible by 10'

we then have:

  • P(A) is 0.5
  • P(B) is 0.1

If we know that x is divisible by 10 (i.e.x is in B) we know that it is also an even number (i.e. x is in A) so P(A|B) = 1.

From Bayes' rule we have:

$$ P(A|B) = \dfrac{P(A \cap B)}{P(B)} $$

note that in our (special) case $P(A \cap B)$, i.e. the probability that x is both an even number and a number divisible by 10 is equal to the probability that x is a number divisible by 10. Therefore we have $P(A \cap B) = P(B)$ and plugging this back into Bayes' rule we get $P(A|B) = P(B) / P(B) = 1$.


For a non-degenerate example consider e.g. A is 'x is divisible by 7' and B is 'x is divisible by 3'. Then P(A|B) is equivalent to 'given that we know that x is divisible by 3 what is the probability that it is (also) divisible by 7 ?'. Or equivalently 'What fraction of the numbers 3, 6, ..., 99 are divisible by 7' ?

$\endgroup$
0
$\begingroup$

I think your initial statement may be a misunderstanding.

You wrote:

The formula for conditional probability of A happening, once B has happened is:

From your phrasing, it may sound as if there are 2 events "First B happened, and then we want to calculate the probability that A will happen".

This is not the case. (The following is valid whether there was a misunderstanding or not).

We have just 1 event, which is described by one of 4 possibilities:

  1. neither $\text{A}$ nor $\text{B}$;

  2. just $\text{A}$, not $\text{B}$;

  3. just $\text{B}$, not $\text{A}$;

  4. both $\text{A}$ and $\text{B}$.

Putting some example numbers on it, let's say$$ \begin{array}{ccccccc} P\left(\text{A}\right) = 0.5, & & P\left(\text{B}\right) = 0.5, & & \text{and} & & \text{A and B are independent}. \end{array} $$

It follows that$$ \begin{array}{ccccc} P\left(\text{A and B}\right)=0.25 & & \text{and} & & P\left(\text{neither A nor B}\right)=0.25. \end{array} $$

Initially (with no knowledge of the event), we knew $P\left(\text{AB}\right) = 0.25$.

But once we know that $\text{B}$ has happened, we are in a different space. $P\left(\text{AB}\right)$ is half of $P\left(\text{B}\right)$ so the probability of $\text{A}$ given $\text{B}$, $P\left(\text{A} \middle| \text{B}\right)$, is $0.5$. It is not $0.25$, knowing that $\text{B}$ has happened.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.