Definition of independent events in probability theory (Wasserman)

Question

In Wasserman's "All of Statistics" p.26 he gives an example of an "independent event" as "flipping a fair coin twice", where the first flip has no effect on the second flip (because the coin has no memory). He then immediately gives the formal definition of independence:

$$\mathbb{P}(A \cap B ) = \mathbb{P}(A)\mathbb{P}(B)$$

where $A$ and $B$ are "events" (subsets of a sample space $\Omega$ with probability $\mathbb{P}(\Omega) = 1$).

He then goes on to show that a "fair die" has independent rolls:

Let $A=\{2,4,6\}$ and $B=\{1,2,3,4\}$.
So $\mathbb{P}(A) = \frac{1}{2}$ and $\mathbb{P}(B) = \frac{2}{3}$.
Then $A \cap B = \{2,4\}$ and $\mathbb{P}(A \cap B) = \frac{1}{3}$.
$A$ and $B$ are independent because $\frac{1}{3} = \frac{1}{2} \times \frac{2}{3}$.

But if we use just a slightly different example, we get a different result:

Let $A=\{2,4,6\}$ and $B=\{1,2,4\}$.
So $\mathbb{P}(A) = \frac{1}{2}$ and $\mathbb{P}(B) = \frac{1}{2}$.
Again, $A \cap B = \{2,4\}$ and $\mathbb{P}(A \cap B) = \frac{1}{3}$.
$A$ and $B$ are not independent because $\frac{1}{3} \ne \frac{1}{2} \times \frac{1}{2}$.

I feel like there's some conflation in terminology here between independent sequential events, and conditional probability; in the first calculation the events are "independent" in the sense $P(A|B) = P(A)$ and $P(B|A) = P(B)$, but in the second case this does not hold. On the other hand, sequential rolls of a dice with a flat probability distribution are indeed "independent events".

Am I misunderstanding what he's trying to show with the example? Can this independence formula apply for sequential events, and if so, how could we notate that?

Answer 1

Your confusion may stem from the definitions of "outcome" vs "event", and from the difference between running a single random experiment vs several independent random experiments.

• A random experiment is an experiment whose outcome is uncertain, in the sense that if the experiment is repeated, the outcome can change. In your example, the random experiment is the roll of a die.
• An outcome $\; \omega \;$ is a result that an experiment can have. In your example, outcomes are 1, 2, 3, 4, 5 and 6.
• The sample space $\; \Omega = \{ \omega_i \} \;$ is the set of all possible events. In your example, $\; \Omega = \{ 1, 2, 3, 4, 5, 6 \} \;$.
• An event $\; A = \{ \omega_1, \omega_2, \ldots, \omega_n \} \;$ is a set of outcomes that we define arbitrarily. In your example, events are $\; \{2, 4, 6 \}$, $\; \{ 1, 2, 3, 4 \}$ and $\; \{ 1, 2, 4 \} \;$.

Once we have defined these concepts, we can ask two different questions. We can either:

• Consider the outcome of a single random experiment, and try to determine whether events of outcomes are dependent or independent.
• Consider the independent outcomes of several independent random experiments, and compute the associated probability.

For example, consider the events of single outcomes when rolling a die, i.e. $\; A_1 = \{ 1 \}$, $\; A_2 = \{ 2 \}$, $\; A_3 = \{ 3 \}$, $\; A_4 = \{ 4 \}$, $\; A_5 = \{ 5 \}$ and $\; A_6 = \{ 6 \}$. We can ask two different questions based on these single-outcome events:

• If we look at one particular die roll, are these events dependent or independent? They are dependent, in the sense that, if you look at one particular die roll, then it is not possible to obtain more than one of these events coexisting at the same time in the same die roll. Either you get a 3, or you get a 4, but it is not possible to get both a 3 and a 4 at the same time. Therefore, each of these events influences the possibility of observing other events in the same roll, so they are dependent. If we look at the probabilistic definition, indeed we see that for $\; A_i, A_j, i \neq j \;$, then $\; P(A_i \cap A_j) = 0 \neq P(A_i) P(A_j) = \frac{1}{36}\;$, which shows that they are not independent.
• If we look at different die rolls, then what is the probability of obtaining first a 1 and then a 3? In this case, the events are independent by definition, so rather than asking the question of whether they are dependent or independent, we are interested in computing probabilities assuming they are independent. This probability is $\; P(1) P(3) = \frac{1}{36} \;$. But note that here we are asking a different question.