What is the intuition behind the formula for conditional probability? - Cross Validated most recent 30 from stats.stackexchange.com 2019-09-23T05:01:04Z https://stats.stackexchange.com/feeds/question/326253 https://creativecommons.org/licenses/by-sa/4.0/rdf https://stats.stackexchange.com/q/326253 30 What is the intuition behind the formula for conditional probability? WorldGov https://stats.stackexchange.com/users/169299 2018-02-01T13:44:30Z 2018-07-21T08:50:48Z <p>The formula for the <a href="https://en.wikipedia.org/wiki/Conditional_probability" rel="nofollow noreferrer"><em>conditional probability</em></a> of $\text{A}$ happening given that $\text{B}$ has happened is:$$P\left(\text{A}~\middle|~\text{B}\right)=\frac{P\left(\text{A} \cap \text{B}\right)}{P\left(\text{B}\right)}.$$</p> <p>My textbook explains the intuition behind this in terms of a Venn diagram.</p> <p><a href="https://i.stack.imgur.com/mPer3.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/mPer3.png" alt="enter image description here"></a></p> <p>Given that $\text{B}$ has occurred, the only way for $\text{A}$ to occur is for the event to fall in the intersection of $\text{A}$ and $\text{B}$. </p> <p>In that case, wouldn't the probability of $P\left(\text{A} \middle| \text{B}\right)$ simply be equal to the probability of $\text{A}$ intersection $\text{B}$, since that's the only way the event could happen? What am I missing? </p> https://stats.stackexchange.com/questions/326253/-/326254#326254 23 Answer by user0 for What is the intuition behind the formula for conditional probability? user0 https://stats.stackexchange.com/users/54664 2018-02-01T13:48:17Z 2018-02-04T02:34:25Z <p>A good intuition is given that B occurred—with or without A—what is the probability of A? I.e., we are now in the universe in which B occurred—the full right circle. In that circle, the probability of A is the area of A intersect B divided by the area of the circle.</p> https://stats.stackexchange.com/questions/326253/-/326276#326276 17 Answer by Hard Core for What is the intuition behind the formula for conditional probability? Hard Core https://stats.stackexchange.com/users/68313 2018-02-01T15:26:38Z 2018-02-03T16:05:07Z <p>I would think of it like this: I take for granted that you understand the intuition until:</p> <blockquote> <p>Given that B has occurred, the only way for A to occur is for the even to fall in the intersection of A &amp; B.</p> </blockquote> <p>and I am going to comment the second image you posted:</p> <ol> <li><p>Imagine that the entire white rectangle is your sample space $\Omega$.</p> <p>Assigning a probability to a set means that you are <strong>measuring</strong> in some sense that set. It is the same as if you measured the area of the rectangle but the probability is a different kind of measure that has specific properties (I won't say anything more about this).</p></li> <li><p>You know that $P(\Omega) = 1$ and this is interpreted like this:</p> <p>$\Omega$ represents all the events that could happen and something has to happen so we have 100% probability that something happens.</p></li> <li><p>Analogously the set $A$ has a probability $P(A)$ that is <strong>proportional</strong> to the probability of the sample space $\Omega$. Graphically speaking you see that $A \subset \Omega$ hence the measure of $A$ (its probability $P(A)$) has to be lesser than $P(\Omega)$. The same reasoning is valid for the set $A \cap B$. This set can be measured and its measure is $P(A \cap B)$.</p></li> <li><p>If now you are told that $B$ has happened you have to think <strong>as if $B$ were your "new" $\Omega$</strong>. If $B$ is your "new" $\Omega$ then you can be 100% sure that everything happens in the set $B$. </p> <p>And what does that mean? It means that now, in the "new" contest $P(B \mid B) =1$, and you have to rescale all the probability measures, taking into account that they have to be expressed in terms of the "new" sample space $B$. It is a simple proportion. </p> <p>Your intuition is almost right when you say that:</p></li> </ol> <blockquote> <p>the probability of P(A | B) would simply be equal to the probability of A intersection B</p> </blockquote> <p>and the "almost" is due to the fact that now your sample space has changed (it is $B$ now) and you want to rescale $P(A \cap B)$ accordingly.</p> <ol start="5"> <li><p>$P(A \mid B)$ is your $P(A \cap B)$ in the new world where the sample space is now $B$. In words you would say it like this (and please try to visualize it on the image with the sets):</p> <p><strong>In the new world the ratio between the measure of $B$ and the measure of $A\cap B$ must be the the same as the ratio between the measure of $\Omega$ and the measure of $A \mid B$</strong></p></li> <li><p>Lastly translate this in mathematical language (a simple proportion):</p></li> </ol> <p>$$P(B):P(A \cap B) = P(\Omega):P(A \mid B)$$</p> <p>and since $P(\Omega)=1$ it follows that:</p> <p>$$P(A \mid B)= P(A \cap B):P(B)$$</p> https://stats.stackexchange.com/questions/326253/-/326284#326284 0 Answer by Haitao Du for What is the intuition behind the formula for conditional probability? Haitao Du https://stats.stackexchange.com/users/113777 2018-02-01T15:45:16Z 2018-02-01T15:55:53Z <p>I feel it is more intuitive when we have a concrete data to estimate the probabilities.</p> <p>Let's use <code>mtcars</code> data as an example, the data looks like this (we only use number of cylinders and transmission type.)</p> <pre><code>&gt; mtcars[,c("am","cyl")] am cyl Mazda RX4 1 6 Mazda RX4 Wag 1 6 Datsun 710 1 4 Hornet 4 Drive 0 6 ... ... Ford Pantera L 1 8 Ferrari Dino 1 6 Maserati Bora 1 8 Volvo 142E 1 4 </code></pre> <p>We can calculate the <strong>joint distribution</strong> on two variables by doing a cross table: </p> <pre><code>&gt; prop.table(table(mtcars$cyl,mtcars$am)) 0 1 4 0.09375 0.25000 6 0.12500 0.09375 8 0.37500 0.06250 </code></pre> <p>The joint probability means we want to consider two variables at the same time. For example, we will ask how many cars are 4 cylinder and manual transmission.</p> <p>Now, we come to conditional probability. I found the most intuitive way to explain conditional probability is using the term <strong>filtering on data.</strong></p> <p>Suppose we want to get $P(am=1|cyl=4)$, we will do following estimations:</p> <pre><code>&gt; cyl_4_cars=subset(mtcars, cyl==4) &gt; prop.table(table(cyl_4_cars$am)) 0 1 0.2727273 0.7272727 </code></pre> <p>This means, we only care cars have 4 cylinder. So we filter data on that. After filtering, we check how many of them are manual transmission.</p> <p>You can compare conditional this with joint I mentioned earlier to feel the differences.</p> https://stats.stackexchange.com/questions/326253/-/326291#326291 5 Answer by Aksakal for What is the intuition behind the formula for conditional probability? Aksakal https://stats.stackexchange.com/users/36041 2018-02-01T16:05:22Z 2018-02-03T01:22:08Z <p>You'll see intuition easily thinking about the following problem.</p> <p>Suppose, you have 10 balls: 6 Black and 4 red. Of Black balls 3 are Awesome and of red balls only 1 is Awesome. How likely it is that a a Black ball is also Awesome?</p> <p>The answer is very easy: it's 50%, because we have 3 Awesome Black balls out of total 6 Black balls.</p> <p>This is how you map probabilities to our problem:</p> <ul> <li>3 balls that are Black AND Awesome correspond to$P(A \cap B)$</li> <li>6 balls that are Black correspond to$P(B)$</li> <li>probability that a ball is Awesome when we KNOW that it is Black:$P(A\mid B)$</li> </ul> https://stats.stackexchange.com/questions/326253/-/326292#326292 3 Answer by kabanus for What is the intuition behind the formula for conditional probability? kabanus https://stats.stackexchange.com/users/192574 2018-02-01T16:06:01Z 2018-02-01T16:06:01Z <p>I would reverse the logic. The probability that both$A$and$B$is either:</p> <ol> <li>The probability$B$happened, and that given that$A$happened. </li> <li>Same but reverse roles for$A$and$B$</li> </ol> <p>This will give you </p> <p>$p(A\cap B)=p(B)p(A\mid B)$</p> <p>If you're looking for a negative to your suggestion, it's while it's true the probability of$A$given$B$is contained in the probability of the product, the space you're rolling the dice in is smaller than your original probability space - you know for sure you're "in"$B$, hence you divide by the size of the new space. </p> https://stats.stackexchange.com/questions/326253/-/326297#326297 1 Answer by Tim for What is the intuition behind the formula for conditional probability? Tim https://stats.stackexchange.com/users/35989 2018-02-01T16:20:56Z 2018-02-01T16:20:56Z <p>Think of it on terms of counts. Marginal probability is how many times A occurred divided by sample size. Joint probability of A and B is how many times A occurred together with B divided by sample size. Conditional probability of A given B is how many times A occurred together with B divided by how many times B occurred, i.e. only the A's "within" B's.</p> <p>You can find nice visual illustration <a href="http://www.countbayesie.com/blog/2015/2/18/bayes-theorem-with-lego" rel="nofollow noreferrer">on this blog</a>, that shows it using Lego blocks.</p> https://stats.stackexchange.com/questions/326253/-/326314#326314 2 Answer by Acccumulation for What is the intuition behind the formula for conditional probability? Acccumulation https://stats.stackexchange.com/users/179204 2018-02-01T17:53:17Z 2018-02-01T17:53:17Z <p>The Venn diagram doesn't represent probability, it represents the measure of subsets of the event space. A probability is the ratio between two measures; the probability of X is the size of "everything that constitutes X" divided the size of "all the events being considered". Any time you're calculating a probability, you need both a "success space" and a "population space". You can't calculated a probability based just on "how big" the success space is. For instance, the probability of rolling a seven with two dice is the number of ways of rolling a seven divided by the total number of ways of rolling two dice. Just knowing the number of ways of rolling a seven is not enough to calculate the probability. P(A|B) is the ratio of the measure of the "both A and B happen" space and the measure of the "B happens" space. That's what the "|" means: it means "make what comes after this the population space".</p> https://stats.stackexchange.com/questions/326253/-/326329#326329 0 Answer by Andre Holzner for What is the intuition behind the formula for conditional probability? Andre Holzner https://stats.stackexchange.com/users/961 2018-02-01T19:41:32Z 2018-02-01T19:54:20Z <p>If <code>A</code> were a superset of <code>B</code> the probability that <code>A</code> happens is always 1 given that <code>B</code> happened, i.e. <code>P(A|B) = 1</code>. However, <code>B</code> itself may have a probability much smaller than 1. </p> <p>Consider the following example: </p> <ul> <li>given <code>x</code> is a natural number in 1..100, </li> <li><code>A</code> is '<code>x</code> is an even number'</li> <li><code>B</code> is '<code>x</code> is divisible by 10' </li> </ul> <p>we then have:</p> <ul> <li><code>P(A)</code> is 0.5</li> <li><code>P(B)</code> is 0.1 </li> </ul> <p>If we know that <code>x</code> is divisible by 10 (i.e.<code>x</code> is in <code>B</code>) we know that it is also an even number (i.e. <code>x</code> is in <code>A</code>) so <code>P(A|B) = 1</code>.</p> <p>From Bayes' rule we have:</p> <p>$$P(A|B) = \dfrac{P(A \cap B)}{P(B)}$$</p> <p>note that in our (special) case$P(A \cap B)$, i.e. the probability that <code>x</code> is both an even number and a number divisible by 10 is equal to the probability that <code>x</code> is a number divisible by 10. Therefore we have$P(A \cap B) = P(B)$and plugging this back into Bayes' rule we get$P(A|B) = P(B) / P(B) = 1$. </p> <hr> <p>For a non-degenerate example consider e.g. <code>A</code> is '<code>x</code> is divisible by 7' and <code>B</code> is '<code>x</code> is divisible by 3'. Then <code>P(A|B)</code> is equivalent to 'given that we know that <code>x</code> is divisible by 3 what is the probability that it is (also) divisible by 7 ?'. Or equivalently 'What fraction of the numbers 3, 6, ..., 99 are divisible by 7' ?</p> https://stats.stackexchange.com/questions/326253/-/326345#326345 2 Answer by glassy for What is the intuition behind the formula for conditional probability? glassy https://stats.stackexchange.com/users/145494 2018-02-01T21:08:26Z 2018-02-04T02:17:49Z <p>I think the best way to think about this is drawing step-by-step paths.</p> <p>Let's describe Event B as rolling a$4$on a fair die - this can be easily shown to have probability$\frac{1}{6}$. Now let's describe Event A as drawing an Ace from a standard 52-card deck of cards - this can be easily shown to have probability$\frac{1}{13}$.</p> <p>Let's now run an experiment where we roll a die and then pick a card. So$P\left(A \middle| B\right)$would be the probability that we draw an Ace, given that we have already rolled a$4$. If you look at the image, this would be the$\frac{1}{6}$path (go up) and then the$\frac{1}{13}$path (go up again).</p> <p>Intuitively, the total probability space is what we have already been given: rolling the$4$. We can ignore the$\frac{1}{13}$and$\frac{12}{13}$the initial down path leads to, since it was GIVEN that we rolled a$4$. By law of multiplication, our total space is then$\left(\frac{1}{6}{\times}\frac{1}{13}\right) + \left(\frac{1}{6}{\times}\frac{12}{13}\right)$.</p> <p>Now what's the probability we drew an Ace, GIVEN that we rolled a$4$? The answer by using the path is$\left(\frac{1}{6} {\times} \frac{1}{13} \right)$, which we then need to divide by the total space. So we get$$P\left(A \middle| B\right) = \frac{\frac{1}{6} {\times} \frac{1}{13}}{\left(\frac{1}{6} {\times} \frac{1}{13}\right) + \left(\frac{1}{6} {\times} \frac{12}{13}\right)}.$$</p> <p><a href="https://i.stack.imgur.com/zsQ1t.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/zsQ1t.png" alt="enter image description here"></a></p> https://stats.stackexchange.com/questions/326253/-/326378#326378 0 Answer by user985366 for What is the intuition behind the formula for conditional probability? user985366 https://stats.stackexchange.com/users/67675 2018-02-02T01:12:46Z 2018-02-04T02:17:32Z <p>I think your initial statement may be a misunderstanding.</p> <p>You wrote:</p> <blockquote> <p>The formula for conditional probability of A happening, once B has happened is:</p> </blockquote> <p>From your phrasing, it may sound as if there are 2 events "First B happened, and then we want to calculate the probability that A will happen".</p> <p>This is not the case. (The following is valid whether there was a misunderstanding or not).</p> <p>We have just 1 event, which is described by one of 4 possibilities:</p> <ol> <li><p>neither$\text{A}$nor$\text{B}$;</p></li> <li><p>just$\text{A}$, not$\text{B}$;</p></li> <li><p>just$\text{B}$, not$\text{A}$;</p></li> <li><p>both$\text{A}$and$\text{B}$.</p></li> </ol> <p>Putting some example numbers on it, let's say$$\begin{array}{ccccccc} P\left(\text{A}\right) = 0.5, &amp; &amp; P\left(\text{B}\right) = 0.5, &amp; &amp; \text{and} &amp; &amp; \text{A and B are independent}. \end{array}$$</p> <p>It follows that$$\begin{array}{ccccc} P\left(\text{A and B}\right)=0.25 &amp; &amp; \text{and} &amp; &amp; P\left(\text{neither A nor B}\right)=0.25. \end{array}$$</p> <p>Initially (with no knowledge of the event), we knew$P\left(\text{AB}\right) = 0.25$.</p> <p>But once we know that$\text{B}$has happened, we are in a different space.$P\left(\text{AB}\right)$is half of$P\left(\text{B}\right)$so the probability of$\text{A}$given$\text{B}$,$P\left(\text{A} \middle| \text{B}\right)$, is$0.5$. It is not$0.25$, knowing that$\text{B}$has happened. </p> https://stats.stackexchange.com/questions/326253/-/326556#326556 4 Answer by Silverfish for What is the intuition behind the formula for conditional probability? Silverfish https://stats.stackexchange.com/users/22228 2018-02-02T22:57:10Z 2018-02-02T22:57:10Z <p>For a basic intuition of the conditional probability formula, I always like using a two way table. Let's say there are 150 students in a yeargroup, of whom 80 are female and 70 male, each of whom must study exactly one language course. The two-way table of students taking different courses is:</p> <pre><code> | French German Italian | Total -------- --------------------------- ------- Male | 30 20 20 | 70 Female | 25 15 40 | 80 -------- --------------------------- ------- Total | 55 35 60 | 150 </code></pre> <p>Given that a student takes the Italian course, what is the probability they are female? Well the Italian course has 60 students, of whom 40 are females studying Italian, so the probability must be:</p> <p>$$P(\text{F|Italian})=\frac{n(\text{F} \cap \text{Italian})}{n(\text{Italian})}=\frac{40}{60}=\frac{2}{3}$$</p> <p>where$n(A)$is the <a href="https://en.wikipedia.org/wiki/Cardinality" rel="nofollow noreferrer">cardinality</a> of the set$A$, i.e. the number of items it contains. Note that we needed to use$n(\text{F} \cap \text{Italian})$in the numerator and not just$n(\text{F})$, because the latter would have included all 80 females, including the other 40 who do not study Italian.</p> <p>But if the question were flipped around, what is the probability that a student takes the Italian course, given that they are female? Then 40 of the 80 female students take the Italian course, so we have:</p> <p>$$P(\text{Italian|F})=\frac{n(\text{Italian} \cap \text{F})}{n(\text{F})}=\frac{40}{80}=\frac{1}{2}$$</p> <p>I hope this provides intuition for why</p> <p>$$P(A|B)=\frac{n(A \cap B)}{n(B)}$$</p> <p>Understanding why the fraction can be written with probabilities instead of cardinalities is a matter of <a href="https://en.wikipedia.org/wiki/Equivalent_fraction" rel="nofollow noreferrer">equivalent fractions</a>. For example, let us return to the probability a student is female given that they are studying Italian. There are 150 students in total, so the probability that a student is female and studies Italian is 40/150 (this is a "joint" probability) and the probability a student studies Italian is 60/150 (this is a "marginal" probability). Note that dividing the joint probability by the marginal probability gives:</p> <p>$$\frac{P(\text{F} \cap \text{Italian})}{P(\text{Italian})}=\frac{40/150}{60/150}=\frac{40}{60}=\frac{n(\text{F} \cap \text{Italian})}{n(\text{Italian})}=P(\text{F|Italian})$$</p> <p>(To see that the fractions are equivalent, multiplying numerator and denominator by 150 removes the "/150" in each.)</p> <p>More generally, if your sampling space$\Omega$has cardinality$n(\Omega)$&mdash; in this example the cardinality was 150 &mdash; we find that</p> <p>$$P(A|B)=\frac{n(A \cap B)}{n(B)}=\frac{n(A \cap B)/n(\Omega)}{n(B)/n(\Omega)}=\frac{P(A \cap B)}{P(B)}$$</p> https://stats.stackexchange.com/questions/326253/-/326712#326712 1 Answer by Michael Le Barbier Grünewald for What is the intuition behind the formula for conditional probability? Michael Le Barbier Grünewald https://stats.stackexchange.com/users/124627 2018-02-04T00:43:49Z 2018-02-04T00:43:49Z <p>At the time of writing there is about 10 answers which seem to all miss the most important point: you are essentially right.</p> <blockquote> <p>In that case, wouldn't the probability of P(A | B) simply be equal to the probability of A intersection B, since that's the only way the event could happen?</p> </blockquote> <p>This is definitely true. This explains why the quantity we to define$P(A\vert B)$is actually$P(A \cap B)$rescaled.</p> <blockquote> <p>What am I missing?</p> </blockquote> <p>You are missing that the probability of B being satisfied given that B is satisfied should be 1 since this is quite a certain event, and not$P(B\cap B) = P(B)$which can well be less than 1. Dividing by$P(B)$makes the conditional probability of B given B equal to 1, as expected. Actually this is even better and makes the map$A \mapsto P(A\vert B)\$ a probability – so a conditional probability is actually a probability.</p>