2
$\begingroup$

So in my intro course to stats, we ecnountered the law of total probability. The definition is $$P(A) = \sum^n_{j=1}P(A\mid H_j)P(H_j)$$ However, the definition says

$$\bigcup^n_{j=1}H_j = S \quad \text{(Union of all Events form Sample Space)}\\ \bigcap^n_{j=1}H_j=\emptyset \quad \text{Events are Pairwise Disjoint}$$

Now I was wondering what happens, when they tell me (in the exam) to calculate the total probability of an event $A$, even though an intersection of H_1 and H_2 exist.

So I constructed the following example and wanted to ask for your help on whether that is correct.

So I constructed two Events $A,B$ with $P(A) = 0.5, P(B) = 0.7 \text{ and } P(A \cap B) = 0.2$

first venn

I then introduce an event $M,$ which can happen according to the following probabilites. $$P(M\mid A \setminus B)=P(M\mid A \cap B)=P(M\mid B \setminus A) = \frac{1}{3}$$

This introduces the next drawing.

second venn

And thus I ultimately calculated $$P(M) = P(A \setminus B\mid M) P(A \setminus B) +P(A \cap B \mid M)P(A \cap B) + P(B \setminus A \mid M) P(B \setminus A) = \frac{1}{3}\cdot(0.3+0.2+0.5)= \frac{1}{3}$$

Is that correct? Is thus the tric to redefine all the events of the subspace as pairwise disjoint? Or is there maybe another way/trick or something one should pay attention to?

$\endgroup$
1
$\begingroup$

This is indeed correct. The law is true if $H_i$ provide a partition of the probability space. Here, you created a new partition with the underlying distributions, but it's still a partition, so the law applies.

$\endgroup$
  • $\begingroup$ Are there cases, when this this redefinition of the partitions is not possible and thus my appraoch not applicable? $\endgroup$ – thebilly Dec 21 '18 at 19:12
  • $\begingroup$ I don't think so. Theoretically you can always redefine the partitions, the difference happens in real life when you are using discrete (i.e. floating point) operations and where you will have numerical noise. But that's the case for any formula. $\endgroup$ – Matthieu Brucher Dec 21 '18 at 19:15
  • $\begingroup$ One case where it is not possible is when you use a partition that is uncountable. In that case the law of total probability above does not apply. $\endgroup$ – Ben Jun 19 at 0:23
1
$\begingroup$

The law of total probability applies when you condition on a partition (i.e., a class of disjoint events that cover the sample space). Given an event $\mathcal{A}$ and a partition $\mathscr{H} = \{ \mathcal{H}_1, \mathcal{H}_2, ..., \mathcal{H}_n \}$ (where the sets $\mathcal{H}_1, \mathcal{H}_2, ..., \mathcal{H}_n$ are disjoint sets with $\cup_{i=1}^n \mathcal{H}_i = \Omega$), the law is derived as follows:

$$\begin{equation} \begin{aligned} \mathbb{P}(\mathcal{A}) = \mathbb{P}(\mathcal{A} \cap \Omega) &= \mathbb{P} \Bigg( \mathcal{A} \cap \bigcup_{i=1}^n \mathcal{H}_i \Bigg) \\[6pt] &= \mathbb{P} \Bigg( \bigcup_{i=1}^n (\mathcal{A} \cap \mathcal{H}_i) \Bigg) \\[6pt] &= \sum_{i=1}^n \mathbb{P} (\mathcal{A} \cap \mathcal{H}_i) \\[6pt] &= \sum_{i=1}^n \frac{\mathbb{P} (\mathcal{A} \cap \mathcal{H}_i)}{\mathbb{P} (\mathcal{H}_i)} \cdot \mathbb{P}(\mathcal{H}_i) \\[6pt] &= \sum_{i=1}^n \mathbb{P}(\mathcal{A} | \mathcal{H}_i) \cdot \mathbb{P}(\mathcal{H}_i). \\[6pt] \end{aligned} \end{equation}$$

The third step follows from the additivity axiom of probability since $(\mathcal{A} \cap \mathcal{H}_i) \cap (\mathcal{A} \cap \mathcal{H}_j) = \varnothing$ for all $i \neq j$. (For simplicity we have assumed here that all the events in the partition have positive marginal probability. If a partition event has zero probability, the conditional probability is undefined, but since it is multiplied by zero anyway, we can define it arbitrarily.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.