11
$\begingroup$

Quite a simple or at least short question: Why is $ \frac{P(A \cap B)}{P(B)} $ divided by $ P(B) $ for the conditional probability?

$ P(A | B) = \frac{P(A \cap B)}{P(B)} $

Random image to visualize: img

I actually would like to make use out of the tree display which I can't grasp at all. Why would I divide by $ P(A+) $ when I had to move along this path? Sorry for the mix-up of $A+$ and $ B $..

img2

$\endgroup$
2
  • 8
    $\begingroup$ It's probably easier to understand from P(A and B) = P(A | B) P(B) $\endgroup$
    – ocram
    Commented Aug 29, 2022 at 15:15
  • $\begingroup$ This lecture gives a couple good ways to think about the intuition of conditional probability. youtu.be/P7NE4WF8j-Q?t=2138 $\endgroup$ Commented Sep 1, 2022 at 11:34

7 Answers 7

7
$\begingroup$

In your tree diagram, if you want to know the probability $P(B+|A+)$ Then you completely ignore the $A-$ section, since you know that that does not occur. So the probability of $B+$ given that $A+$ occurred is the count of the ways $A+B+$ can occur divided by the total number of ways that $A+$ can occur, that is the combination of $A+B+$ and $A+B-$, which together is $A+$, so you would need to divide by the count (or probability) of $A+$ happening.

It might be easier to think of actual events that you can count the ways it happens.

One example:

Roll a fair die (possible outcomes are 1, 2, 3, 4, 5, 6)

A is the event that the roll is greater than 3 (4, 5, 6)

B is the event that an odd number was rolled (1, 3, 5)

So the probability of A and B is $\frac16$ since 5 is the only number that matches, but if we know that A happened, then $P(B|A) = \frac13$ since we know that the outcome was 4, 5, or 6. We can get there by counting (1 out of 3 possibilities) or by the math with probabilities $\frac16$ divided by $\frac12$. If you do not divide by what you are conditioning on, then you are looking at the joint probability, not the conditional probability. Also think about the Venn diagram. You need to divide by the area of the circle that you are conditioning on so that the total area of the sum of the conditional probabilities is 1.

$\endgroup$
7
  • $\begingroup$ Thank you! The other answers maybe even more illustrative but this one matches my understand/perception the best :) $\endgroup$
    – Ben
    Commented Aug 30, 2022 at 6:49
  • $\begingroup$ I've a follow up question: When $A+$ and $B*$ are the probability of both events happening (means $A+ \cdot B+$), then I consider already the probability of $A+$ or not? $\endgroup$
    – Ben
    Commented Aug 30, 2022 at 9:05
  • $\begingroup$ I really like this explanation, because it is an easy transition between this "tree" model and the "geometric" model demonstrated by whuber. $\endgroup$ Commented Aug 30, 2022 at 13:47
  • 1
    $\begingroup$ @Ben, Not sure exactly what you question/notation means. The probability of A+ and B+ is only the product of the 2 probabilities if A and B are independent (then the conditional probability is the same as the unconditional). The probability of A+ contains both the probability of both A+ and B+ and the probability of A+ and B-. Think about the following probabilities from rolling a fair die. What is the probability that the number shown is both even and a 6? what is the probability that it is 6 given that it is even? what is the probability that it is even given it is a 6? $\endgroup$
    – Greg Snow
    Commented Aug 30, 2022 at 19:22
  • 1
    $\begingroup$ @Ben, yes the joint probability is the product of the marginal probabilities only when they are independent. $\endgroup$
    – Greg Snow
    Commented Aug 31, 2022 at 14:32
15
$\begingroup$

The tickets-in-a-box model provides a helpful picture. It makes the formula for conditional probability immediate and obvious.

Let $\mathcal{A}$ and $\mathcal{B}$ designate events: collections of types of tickets. (That is, whenever a ticket for a particular outcome is in an event, all the tickets for that outcome must be in the event.) Recall that the probability of $\mathcal A$ is defined as the proportion of tickets of type $\mathcal A$ in the box. It is a ratio of that ticket count to the count of all tickets in the box $\Omega.$ (Some people insist all probabilities are conditional and to emphasize this would write the probability as $\Pr(\mathcal A\mid \Omega).$)

If we were to remove all tickets not of type $\mathcal{B},$ we would still have a box with tickets -- all of type $\mathcal B,$ obviously. Assuming some tickets remain (that is, $\mathcal{B}$ is not empty), this is still a tickets-in-a-box model.

The selective removal can alter the relative proportions of $\mathcal A$ among the remaining tickets. The new proportion of tickets of type $\mathcal{A}$ is their probability when $\mathcal B$ is viewed as the "box" -- the sample space. It is called the conditional probability of $\mathcal{A},$ conditional on $\mathcal{B},$ and is written $\Pr{\left(\mathcal{A}\mid\mathcal{B}\right)}.$

enter image description here

In this Venn diagram, areas (relative to the total area of the rectangle $\Omega$ representing the whole probability space) represent probabilities. It looks like both the events $\mathcal{A}$ and $\mathcal{B}$ have probabilities around $1/3$ each. But when all non-$\mathcal{B}$ points are removed, the new sample space is $\mathcal{B}$ itself and the portion of $\mathcal{A}$ within it is tiny – just the sliver of space where the circles intersect, as a proportion of the area of the circle representing $\mathcal{B}.$ In this example $\Pr{\left(\mathcal{A}\right)}\approx1/3$ but $\Pr{\left(\mathcal{A}\mid\mathcal{B}\right)}\approx1/100,$ exemplifying how dramatically the conditioning on $\mathcal B$ can change the probabilities.

It should be apparent that (a) this construction works provided $\mathcal{B}$ does not have zero probability and (b) in that case, the new relative area of $\mathcal A$ is $$Pr{\left(\mathcal{A}\mid\mathcal{B}\right)=}\frac{\Pr{\left(\mathcal{A}\cap\mathcal{B}\right\}}}{\Pr{\left(\mathcal{B}\right)}}.$$


The tree diagram is merely another way to (partially) represent the Venn diagram. Branching at $\mathcal A$ corresponds to slicing everything in $\Omega$ into outcomes in $\mathcal A$ and outcomes not in $\mathcal A;$ likewise, branching at $\mathcal B$ slices everything by $\mathcal B.$

In your diagram you condition on $\mathcal A.$ This means removing everything from the tree that is outside $\mathcal A.$ Afterwards, only two leaves remain: everything in $\mathcal A$ and in $\mathcal B$ ($\mathcal A \cap \mathcal B$) and everything in $\mathcal A$ and not in $\mathcal B$ ($\mathcal A \cap (\Omega \setminus \mathcal B)$). Those leaves represent all the tickets remaining in the box. Trees are usually decorated with probabilities permitting you to compute the relative proportions of those leaves.

Translating between different ways of visualizing the same thing is a useful skill because some visualizations support specific concepts or calculations especially well, and others shine in other areas.

$\endgroup$
11
$\begingroup$

$$ P(A \cap B) = P(A \cap B) \\ P(A | B) P(B) = P(A \cap B) \\ P(A|B) = \frac{P(A \cap B)}{P(B)} $$

$\endgroup$
2
  • 2
    $\begingroup$ This just raises the question, why is $P(A | B) P(B) = P(A \cap B)$ true? $\endgroup$
    – chepner
    Commented Sep 1, 2022 at 14:21
  • $\begingroup$ Some authors, such as Bruno de Finetti, introduce this as an axiom of probability. Likewise, the Kolmogorov answers are employing definitions of conditional probability, sometimes illustrated intuitively with tree diagrams or box ticket models. $\endgroup$
    – Sycorax
    Commented Sep 1, 2022 at 15:00
4
$\begingroup$

Sycorax has answered it nicely. But let me leave behind an intuitive set up that articulates the genesis of the concept.


Start with the very familiar experiment: throwing a die. Let $A:=\{\text{odd number appears}\}, ~ B:=\{\text{a digit smaller than or equal three appears}\}.$ One might wonder what the probability would be of $A$ had it already been assured that $B $ was realised.

Construct the probability space $(\Omega, \mathcal A, \mathbb P) ,$ with $\Omega= \{1, 2,3,4,5,6\}, ~\mathcal A= 2^\Omega$ and $\mathbb P$ being the discrete uniform distribution on $\Omega.$ Naturally $A=\{1, 3,5\}, ~B=\{1, 2,3\}.$ If $B$ has indeed occurred, then it is plausible to assume the uniform distribution on the remaining possible outcomes, i.e. $\{1,2,3\}.$ For assessing the new situation, define a new probability measure $\mathbb P_B$ on $(B, 2^B) $ by

$$ \mathbb P_B[C]:= \frac{\#C}{\#B}, ~C\subset B;$$ natural extension would be to assign probability $0$ on $\Omega\setminus B$ as

$$\mathbb P[C|B]:= \mathbb P_B[C\cap B]:= \frac{\#(C\cap B) }{\#B}, ~C\subset \Omega.$$

In that case $\mathbb P[A|B] =\frac{\# \{1,3\}}{\#\{1,2,3\}} = \frac23.$ This is different from $\mathbb P[A]=\frac12.$

This prompts us to define the conditional probability for any $B\in \mathcal A$ as a new probability measure $\mathbb P[\cdot|B]$ on the space $(\Omega, \mathcal A)$ for $\mathbb P[B] > 0$ as

$$ \mathbb P[A|B] =\begin{cases}\frac{\mathbb P[A\cap B]}{\mathbb P[B]},~~ \mathbb P[B] > 0\\ 0 ,~~~~~~~~~~~\textrm{otherwise}\end{cases}.$$


Reference

Probability Theory: A Comprehensive Course, Achim Klenke, Springer, 2014.


After edit of OP:

As the definition developed above, that $A^+$ has occured provides new information has been incorporated by dividing by $\mathbb P[A^+].$

$\endgroup$
2
$\begingroup$

A more intuitive way to understand is to know that conditioning on an event with strictly positive probability defines a new probability measure.

So instead of looking at the bigger picture (the original sample space, say $X$), we are now restricting to a possibly much smaller sample space (which is $B$ in your case). The intersection $A \cap B$ will be smaller in probability with $X$ being the sample space, but larger when we take the $B$ to be the whole sample space. So we adjust for restricting to a smaller sample space by enlarging the probability of the intersection $A\cap B$. This normalizing process is done by dividing by $\mathbb{P}(B)$. Which is easy to understand, we are thus calculating the relative area of $A\cap B$ with respect to $B$, instead of $X$.

You can also view $\mathbb{P}(A)$ as a conditional probability by noting that $$ \mathbb{P}(A)=\mathbb{P}(A\cap X) = \dfrac{\mathbb{P}(A\cap X)}{\mathbb{P}(X)},$$ where $\mathbb{P}(X)=1$.

$\endgroup$
1
$\begingroup$

First of all, I disagree with the picture. Blue set does not refer to $P(A)$ but $P(A\cap B')$. Green set refers to $P(B\cap A')$.

Back to the question: Why $$ P(B^+|A^+)=\frac{P(B^+\cap A^+)}{P(A^+)}? $$

We can understand it by saying $P(B^+|A^+)$ as the probability that event $B^+$ happening given that event $A^+$ happened.

It means that, if I you have information that $A^+$ has happened,

  1. you must make sure that both events happened at the same time. Hence, you have $P(B^+\cap A^+)$.
  2. your answers must be with respect to the event $A^+$. Your answers are constrained to event $A^+$. Hence, you have a ratio to $P(A^+)$.

As an example, let $A^+$ be the event someone goes outside to play and $B^+$ be the event that someone will come home after 6pm. If you know that someone has already went outside to play, what is their probability to come home after 6pm? You already know that $A^+$ happened, so you will have

  1. that person went outside to play AND that person will come home after 6pm.
  2. that the probability above happened with respect to that person will come home after 6pm.
$\endgroup$
1
$\begingroup$

Intuitive, hand-wavey explanation that I feel is lacking here. (This argument works 100% fine in the case of uniform probability with a finite number of outcomes, such throwing a fair die, or picking cards from a well-shuffled deck, but to work more generally it would need some rephrasing.)

The first probability formula you learn is $$ P(\text{Something}) = \frac{\text{Number of outcomes where Something happens}}{\text{Total number of relevant outcomes}} $$ We are going to use this three times.

First off, the probability $P(B^+\mid A^+)$ is given by $$ P(B^+\mid A^+) = \frac{\text{Number of outcomes where }B^+\text{ and } A^+\text{ happen}}{\text{Number of outcomes where }A^+\text{ happens}} $$ Now we divide numerator and denominator by the total number of outcomes. This does not change the value of the fraction: $$ = \frac{\text{Number of outcomes where }B^+\text{ and } A^+\text{ happen}/\text{Total number of outcomes}}{\text{Number of outcomes where }A^+\text{ happens}/\text{Total number of outcomes}} $$ Applying our basic probability formula again, to numerator and denominator separately, we do indeed get $$ = \frac{P(A^+\cap B^+)}{P(A^+)} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.