I'm currently trying to better understand the fundamental notions of statistics.

Navigating through multiple sites, I've found this formula for joint probability. P(A ∩ B) = P(A)*P(B)

Namely, if we want to know what is the probability of A and B to co-occur, we just multiply their individual (marginal) probabilities.

As for the conditional probability, the formula stands like this: P(A | B) = P(A∩B) / P(B)

So, if we want to know the probability of A when we already know B, we divide the joint probability of A and B by the probability of B.

If we develop this formula, we will get: P(A | B) = P(A)*P(B)/P(B)

Further, we can simplify the fraction by dividing it by P(B), and we are left with P(A | B) = P(A)

Suppose we want to estimate the probability of drawing from a deck of cards a red colored card of 4 (hearts or diamonds), while already knowing it's colored red.

Individual (marginal) probabilities

P(red) = 1/2 = 0.5 P(4) = 4/52 = 1/13

Joint Probability (to draw a red 4)

P(4 ∩ red) = P(A)*P(B) = 1/2 * 1/13 = 1/26

Conditional probability (prob to draw a 4 already knowing it is red)

P(4 | red) = P(4 ∩ red) / P(red) = P(4)*P(red)/P(red) P(4 | red) = 1/13*0.5/0.5 = 1/13

If we know that the card we are about to draw is red, the probability of it being a red 4 is 1/13 (or 7.6%), which is just the probability of drawing a 4 from the deck. This makes sense because we have eliminated half of the possibilities (black cards), and we are actually drawing from a set of 26 cards that has 2 fours in it. So, 2 out of 26 (or 1 of 13) of the drawn cards will be a red 4.

The results look fine, but it really seems that I am missing something, because in the relationship P(A | B) = P(A) doesn't take into account the probability of B.

I will be highly grateful is someone could look into my question and give a response.

Thanks in advance!

  • $\begingroup$ Check out this question. Your probability for the intersection isn't always right. $\endgroup$ – CzechInk Feb 9 '20 at 22:37
  • 1
    $\begingroup$ All clear, thank you all for the explanations. $\endgroup$ – Stefan Feb 18 '20 at 17:13

$P(A\cap B)=P(A)P(B)$ is true if and only if $A$ and $B$ are independent. So, $P(A|B)\neq P(A)$ in general. In your example, it holds because the events are fortunately independent. You already informally found it in your reasoning paragraph, by thinking about the sample space.

For a simple counter example, consider the case where $A=B$. $P(A|B)=1$ clearly, because if you know that $B=A$ happened, the probability of $A$ happening is $1$.

  • $\begingroup$ Typo in the last sentence: because if you know that A happened. It's B, obviously. $\endgroup$ – Rui Barradas Feb 10 '20 at 16:46
  • $\begingroup$ @RuiBarradas It wasn't meant to be a typo, but maybe this is more clear. In the reason part, I was just using the fact that $B=A$ to make sense. $\endgroup$ – gunes Feb 10 '20 at 16:50
  • $\begingroup$ Yes, it is, thanks. $\endgroup$ – Rui Barradas Feb 10 '20 at 16:52

P(A ∩ B) = P(A)*P(B) this expression is true only when events A and B are independent. It is not true in general.


It might help to think of P(A|B) as meaning "What percentage of B is A", and to think of all probabilities as conditional; if a probability is not given explicitly as conditional, there's an implicit universal condition; P(A) can be taken as P(A|S) where S is the entire set you're considering. In this case, S is just "any card". So if A is "card is red" and B is "card is a 4", then P(A|S) is "probability that what we draw is red, given that it's from a standard deck" or "percentage of all the cards that are red". P(A|B) is "percentage of all the 4s that are red". P(A ∩ B) = P(A ∩ B|S) = percentage of cards that are red and 4s.

So when you ask "Is P(A) = P(A|B)", you're asking "Is there percentage of all cards that are red equal to the percentage of 4s that are red?" In this case, the answer is "yes". However, this is not in general the case. The property of these percentages being equal is known as "independence".

It's easy to come up with examples of variables that are dependent. For instance, P(Diamond|Red) != P(Diamond). Any time a subpopulation has a property at a rate different from the general population, the conditional probability is different. E.g. P(child goes to a private school|parents make more than $100k) != P(child goes to private school).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.