Bayes' theorem: $$ P(A|B) = \frac{P(B|A)P(A)}{P(B)}. $$

Clearly, $P(B)>0$ is required. However, $$ P(B|A) := \frac{P(B \cap A)}{P(A)}, $$ so if $P(A)=0$ we would have $$ P(A|B) = \frac{\frac{P(B \cap A)}{0}0}{P(B)}, $$ which is undefined.

So, even though I usually see Bayes' theorem written with the condition $P(B)>0$, it seems we also need $P(A)>0$. Am I missing something?


You're not missing anything!

Read the formula literally: $\operatorname{Pr}\left(A\ |\ B\right)$ is the fraction of $\operatorname{Pr}\left(B\right)$ that is contributed by $\operatorname{Pr}\left(A \cap B\right)$. This is what the Venn diagram in Hamed's answer depicts.

If $\operatorname{Pr}\left(B\right)$ is zero, there's nothing to contribute to. Think of it this way: Jill has \$0. What fraction of that \$0 did Jack contribute? The fraction is undefined.

That's what Xi'an was saying in his comment.

  • 2
    $\begingroup$ The literal reading of the formula must be in terms of mathematical definitions. Your reading is a good intuition for finite sample spaces, but it fails--and gives an incorrect conclusion--otherwise. For instance, according to this answer it would not be legitimate to write about the conditional probability of the response in the bivariate normal model of regression (which is a fundamental tool) because $P(B=b) = 0$ when $B$ has a continuous distribution. $\endgroup$ – whuber Jun 26 '15 at 17:49
  • 1
    $\begingroup$ @whuber I don't see the issue. Isn't it not legitimate to write about $Pr(Y|X)$ for exactly that reason? $\endgroup$ – shadowtalker Jun 26 '15 at 18:00
  • 1
    $\begingroup$ See math.uah.edu/stat/expect/Conditional2.html for a nice introduction to conditional probability. $\endgroup$ – whuber Jun 26 '15 at 18:01
  • $\begingroup$ @whuber you might be confused because I wrote a confusing answer: I misread the question. I just realized the question is "why is it necessary to have $P(A) > 0$" and not "why is it necessary to have $P(B) > 0$". $\endgroup$ – shadowtalker Jun 26 '15 at 20:18
  • $\begingroup$ That said, I still don't understand your comment, and I don't see how my explanation fails to apply to continuous sample spaces. Maybe you can move this to chat to clarify, and then I can delete the answer $\endgroup$ – shadowtalker Jun 26 '15 at 20:26

Conditional probability actually confines the space to the smaller space. Let give you an example. Assume you are looking for the probability $P(B|A)$. That is the probability of red section in plot to probability of $A$ (sum of yellow and red section), $P(B|A)=P(B\cap A)/P(A)$. So assume $A=\phi$ then $B\cap \phi=\phi$ and $P(B\cap \phi)=P(\phi)=0$ and $P(A)=0$. Then probability is not defined.

For the second part of your question, we need to consider that conditional probability is defined if denominator is non-zero(positive). That is the formula $P(A|B)=\frac{P(A\cap B)}{P(B)}$ is defined if $p(B)>0$ then assuming that $P(A)=0$ in your case is not true. I hope I could explain that clearly.

enter image description here

  • $\begingroup$ Thanks for the nice response. I can only cancel $P(A)$ if $P(A) \neq 0$, though. $\endgroup$ – bcf Jun 26 '15 at 20:16
  • $\begingroup$ as I updated the post, we need to consider that we cannot write conditional probability without assuming that denominator is positive. that means if we write the formula, that means that we have accepted that fact that denominator is positive. Then in your case $P(A)=0$ is not true. thanks $\endgroup$ – TPArrow Jun 26 '15 at 23:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.