I've got a question regarding the different variations of the denominator in Bayes' Theorem. I'm led to believe the denominator of this equation is = P(B)

enter image description here

I did some research and read about a rule pertaining to the 3rd axiom stating P(A) = summation of P(Ai,Bj) of mutually exclusive events. I also know P(A,B) = to P(B|A) * P(A). We can all agree though that if both events are mutually exclusive P(A,B) = 0. So wouldn't that just sum a bunch of 0's? Can someone please help explain this?


The way the sum rule of probability is generally stated is, $$ P(A) = \sum_B P(A,B) $$

where the sum is over all possible mutually exclusive values of the variable $B$. Not mutually exclusive with $A$, but mutually exclusive with each other. So, if $b_k$ and $b_l$ are two possible values of $B$ then they are mutually exclusive with each other. That is, you cannot have $b_k$ and $b_l$ both in one observation if $k \ne l.$ But there's no assumption on either of their likelihood of appearing with any observation of $A$.

As an illustrative example, suppose you throw a dart at a dart board. The position on which the dart lands can be described by the radius, $r$, from the center, and the angle $\phi \in [0, 2 \pi)$. The probability distribution of its landing spot can be written as $P(r, \phi).$ Clearly you can't land on two different values of $\phi$, nor can you land on different values of $r.$ But there's no apparent restriction on a value of $r$, given a value of $\phi$, unless explicitly stated in the joint distribution. To get the probability distribution of the radius, we use the sum rule, summing the joint distribution over all values of $\phi$, $$ P(r) = \int_0^{2 \pi} P(r, \phi) d \phi. $$

  • $\begingroup$ Perfect. Thank you for the explanation that clears it up. $\endgroup$ – Jared B May 9 '17 at 1:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.