4
$\begingroup$

I undersrand that a probability space is composed of a sample space $\Omega$ containing all possible outcomes of an experiment, an event space $\mathcal F$ containing all events of interest (e.g. the power set of $\Omega$) and a probability measure $\mathbb P$ that assigns a probability $p$ to each event in $\mathcal F$.

  • We say two events $A$ and $B$ in $\mathcal F$ are disjoint iff $A\cap B = \emptyset$

  • On the other hand, two events are said to be independent iff $\mathbb P(A\cap B) = \mathbb P(A) \mathbb P(B)$. Meaning that additional knowledge of one event does not change the probability of occurrence of the other event. But I can't seem to fully wrap my mind around this. What does independent mean here? How can two events have a nonzero intersection and yet have nothing to do with one another? Some people say it means the state space can be factorized into a cartesian product, which I don't really get.

  • For example: given $\Omega =\{1,2,3,4\}$ how can one intuitively realise that the events $A=\{1,2\}$, $B=\{1,3\}$ and $C=\{1,4\}$ are only pairwise independent and not independent?

$\endgroup$
  • $\begingroup$ Is something missing from the last paragraph - determining whether the events are pairwise dependent would require specifying $\mathbb{P}$? $\endgroup$ – Juho Kokkala Sep 22 '16 at 16:23
  • 2
    $\begingroup$ I think we're assuming the distribution is uniform. $\endgroup$ – dsaxton Sep 22 '16 at 16:46
4
$\begingroup$

It can be helpful to think of the definition of conditional probability when we try to understand the definition of independence. Recall that $$ P(A \mid B) = \frac{P(A \cap B)}{P(B)} $$ for $B$ with $P(B) > 0$. If $A$ and $B$ are independent, which means $B$ tells us nothing about $A$, then the right hand side should reduce to $P(A)$. If we plug in $P(A \cap B) = P(A) P(B)$ then $P(B)$ simply cancels and the right side does indeed become $P(A)$. But the nice thing about defining independence in this way is that we can talk about independence of events even when conditional probabilities are undefined (which happens whenever the conditioning event has probability zero).

Another way to think about is to view probabilities as fractions of a sample space. If $B$ happens then we're left with $P(B)$ of the sample space remaining. Now if $A$ also happens then $B$ just becomes the new sample space and we reduce it by a factor of $P(A)$ just as we would if we were reducing the original sample space. So in a certain sense $A$ is "blind" to the occurrence of the event $B$, which is more or less what independence means.

Disjointness on the either hand is a very different concept. It says that $A$ and $B$ cannot simultaneously occur, so they are very much dependent (this is strictly speaking not true, can you find a counterexample?).

Regarding your last question, if we look at the sample space $\Omega = \{1, 2, 3, 4 \}$ and the events $A = \{1, 2 \}$, $B = \{1, 3 \}$ and $C = \{1, 4 \}$ we can start by supposing the event $A \cap B = \{ 1 \}$ has occurred. Then $C$ has no choice but to occur as well, so it isn't as though when $C$ occurs here we begin with a sample space and then reduce it to a proper subset whose relative size is $P(C)$, as would be the case if $C$ was independent of $A \cap B$. The new reduced sample space is now a subset of $C$, and $C$ always occurs. The fact that these events happen to be pairwise independent is mostly a contrived property of the problem, and when we talk about independence we almost always mean mutual independence, where no event has any knowledge of occurrence of the others.

$\endgroup$
  • $\begingroup$ Many many thanks dsaxton. Only bit im still struggling with is to understand the sample space reduction and its repercussion. So if i understood correctlty: here by reduced sample space we mean all events that contain at least the element $\{1\}$? Really trying to understand this size proportionality of sample space and uts relation to probability of an event :( $\endgroup$ – user929304 Sep 22 '16 at 15:30
  • $\begingroup$ When $A$ and $B$ both occur then $\{ 1 \}$ is the event that has occurred, not merely an event containing $\{ 1 \}$. But $C$ is the event $\{ 1, 4\}$ (read this as "one or four") which of course has happened if the event $\{ 1 \}$ happened. $\endgroup$ – dsaxton Sep 22 '16 at 16:44
3
$\begingroup$

Meaning that additional knowledge of one event does not change the probability of occurrence of the other event. But I can't seem to fully wrap my mind around this. What does independent mean here? How can two events have a nonzero intersection and yet have nothing to do with one another?

"two events have nothing to do with one another" means that both could occur simultaneously, or one could occur but not the other, or both might not occur. If two events have zero intersection, then they very much do have something to do with each other. If we know (or we are told or we assume) that one event has occurred, then we know that the other event cannot possibly have occurred. (Remember that an event (a collection of outcomes) is said to have occurred on a trial if the outcome of the trial is a member of that event). If $A\cap B = \emptyset$, then whenever the outcome $x \in A$, we know for sure that $x \notin B$. In short, the occurrence of $A$ tells us for sure that $B$ did not occur, but instead $B^c$ has occurred. Thus, $P(B\mid A) = 0$, $P(B^c\mid A) = 1$. Unless $P(B)$ happens to have value $0$, it is clear that knowledge of the occurrence of $A$ has indeed "changed the probability of the occurrence of $B$."

Thus, at the very least, I hope that you can disabuse yourself of the notion that "nothing to do with one another" means disjoint. But, just having $P(A\cap B)$, $P(A\cap B^c)$, $P(A^c \cap B)$, and $P(A^c \cap B^c)$ all be nonzero is not sufficient to assert independence. As pointed out by @dsaxton, what is needed to assert independence is $$P(B\mid A) = P(B),$$ that is, "knowledge of one event does not change the probability of occurrence of the other event": the conditional probability that $B$ occurs conditioned on the occurrence of $A$ is the same as the unconditional probability.

For a more extensive discussion of these matters as well as an answer to the query posed in the penultimate paragraph of dsaxton's answer, I refer you to this previous answer of mine from four years ago.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.