2
$\begingroup$

The total probability theorem states the following:

Let $A_1$,...,$A_n$ be a partition of $\Omega$. For any event B,

$$Pr(B)=\sum_{i=1}^n Pr(A_i)Pr(B|A_i)$$

We know that $Pr(B|A_i)= \frac{Pr(B\cap A_i)}{Pr(A_i)}$, therefore the $Pr(A_i)Pr(B|A_i)$ can simplify to $Pr(B\cap A_i)$.

Couldn't the above theorem then be directly expressed as:

$$Pr(B)=\sum_{i=1}^n Pr(A_i\cap B)$$

If yes, then why use the above representation?

$\endgroup$
  • $\begingroup$ Why not? The former is in terms of conditional probabilities, the latter in terms of disjoint sets; conceptually it may be easier to relate to one or the other, depending upon your background. $\endgroup$ – jbowman Jul 17 at 16:33
  • $\begingroup$ I guess because it seemed redundant to me to see this expression given that there is a shorter way to present it. Particularly, if you do the calculation by hand you immediately end up simplifying the expression and only computing the intersection. I was wondering if there was anything else beyond–potentially– convention. $\endgroup$ – aedcv Jul 17 at 16:42
  • $\begingroup$ Well, in many applications, it's useful to think of it in terms of conditional probabilities because that's what you have, but other than that, you're right, they are equivalent. $\endgroup$ – jbowman Jul 17 at 16:45
7
$\begingroup$

Yes, the theorem of total probability can be expressed as $$P(B) = \sum_n P(B\cap A_n)\tag{1}$$ (which form you seem to prefer) instead of the more usual $$P(B) = \sum_n P(A_n)P(B\mid A_n)\tag{2}$$ but the whole point of expressing $P(B)$ in the form $(2)$ is to remind the reader that $P(B)$, the unconditional probability of $B$, is just a weighted sum of the conditional probabilities $P(B\mid A_n)$. The weights are, of course, just the $P(A_n)$ which sum to $1$. Put another way, the unconditional probability $P(B)$ is the average value (or expected value) of the (conditional) probability of $B$ given that various (mutually exclusive) conditions (the events $A_n$) have occurred. From this we can deduce that the unconditional probability $P(B)$ is necessarily bounded above by $\max P(B\mid A_n)$ and bounded below by $\min P(B\mid A_n)$. So, $(2)$ does have its uses. In my opinion, it provides more insight into the issues than $(1)$.

$\endgroup$
2
$\begingroup$

I guess the main reason is the immediate applicability to the elementary version of Bayes' Theorem, in which (i) sets $A_i,$ for $i = 1, \dots k$ form a partition of the sample space, (ii) $P(A_i)$ and $P(B|A_i)$ are known, and (iii) $P(A_1|B)$ is to be computed. [One of the applications mentioned by @jbowman.]

Then $$P(A_1|B) = \frac{P(BA_1)}{P(B)} = \frac{P(A_1)P(B|A_i)}{\sum_{i=1}^k P(A_i)P(B|A_i)}.$$

$\endgroup$
  • $\begingroup$ To my mind, an important take-away from this form of Bayes' formula is that the numerator is just one of the terms in the denominator, and thus need not be computed again. While grading innumerable exams, I was always surprised by the number of students whose computation of $P(A_1)P(B\mid A_1)$ while computing the numerator differed from the computation of the same quantity while figuring out the denominator. $\endgroup$ – Dilip Sarwate Jul 17 at 18:32
  • $\begingroup$ @DilipSarwate. Oh yes, seen some of those myself. $\endgroup$ – BruceET Jul 17 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.