# Definition of independent events in probability theory (Wasserman)

In Wasserman's "All of Statistics" p.26 he gives an example of an "independent event" as "flipping a fair coin twice", where the first flip has no effect on the second flip (because the coin has no memory). He then immediately gives the formal definition of independence:

$$\mathbb{P}(A \cap B ) = \mathbb{P}(A)\mathbb{P}(B)$$

where $$A$$ and $$B$$ are "events" (subsets of a sample space $$\Omega$$ with probability $$\mathbb{P}(\Omega) = 1$$).

He then goes on to show that a "fair die" has independent rolls:

Let $$A=\{2,4,6\}$$ and $$B=\{1,2,3,4\}$$.
So $$\mathbb{P}(A) = \frac{1}{2}$$ and $$\mathbb{P}(B) = \frac{2}{3}$$.
Then $$A \cap B = \{2,4\}$$ and $$\mathbb{P}(A \cap B) = \frac{1}{3}$$.
$$A$$ and $$B$$ are independent because $$\frac{1}{3} = \frac{1}{2} \times \frac{2}{3}$$.

But if we use just a slightly different example, we get a different result:

Let $$A=\{2,4,6\}$$ and $$B=\{1,2,4\}$$.
So $$\mathbb{P}(A) = \frac{1}{2}$$ and $$\mathbb{P}(B) = \frac{1}{2}$$.
Again, $$A \cap B = \{2,4\}$$ and $$\mathbb{P}(A \cap B) = \frac{1}{3}$$.
$$A$$ and $$B$$ are not independent because $$\frac{1}{3} \ne \frac{1}{2} \times \frac{1}{2}$$.

I feel like there's some conflation in terminology here between independent sequential events, and conditional probability; in the first calculation the events are "independent" in the sense $$P(A|B) = P(A)$$ and $$P(B|A) = P(B)$$, but in the second case this does not hold. On the other hand, sequential rolls of a dice with a flat probability distribution are indeed "independent events".

Am I misunderstanding what he's trying to show with the example? Can this independence formula apply for sequential events, and if so, how could we notate that?

• The braces $\{,\}$ are set notation: they do not indicate sequences. Apart from the motivational example of two flips of a fair coin, there are no sequences involved in your question.
– whuber
Feb 7, 2021 at 15:48

Your confusion may stem from the definitions of "outcome" vs "event", and from the difference between running a single random experiment vs several independent random experiments.

• A random experiment is an experiment whose outcome is uncertain, in the sense that if the experiment is repeated, the outcome can change. In your example, the random experiment is the roll of a die.
• An outcome $$\; \omega \;$$ is a result that an experiment can have. In your example, outcomes are 1, 2, 3, 4, 5 and 6.
• The sample space $$\; \Omega = \{ \omega_i \} \;$$ is the set of all possible events. In your example, $$\; \Omega = \{ 1, 2, 3, 4, 5, 6 \} \;$$.
• An event $$\; A = \{ \omega_1, \omega_2, \ldots, \omega_n \} \;$$ is a set of outcomes that we define arbitrarily. In your example, events are $$\; \{2, 4, 6 \}$$, $$\; \{ 1, 2, 3, 4 \}$$ and $$\; \{ 1, 2, 4 \} \;$$.

Once we have defined these concepts, we can ask two different questions. We can either:

• Consider the outcome of a single random experiment, and try to determine whether events of outcomes are dependent or independent.
• Consider the independent outcomes of several independent random experiments, and compute the associated probability.

For example, consider the events of single outcomes when rolling a die, i.e. $$\; A_1 = \{ 1 \}$$, $$\; A_2 = \{ 2 \}$$, $$\; A_3 = \{ 3 \}$$, $$\; A_4 = \{ 4 \}$$, $$\; A_5 = \{ 5 \}$$ and $$\; A_6 = \{ 6 \}$$. We can ask two different questions based on these single-outcome events:

• If we look at one particular die roll, are these events dependent or independent? They are dependent, in the sense that, if you look at one particular die roll, then it is not possible to obtain more than one of these events coexisting at the same time in the same die roll. Either you get a 3, or you get a 4, but it is not possible to get both a 3 and a 4 at the same time. Therefore, each of these events influences the possibility of observing other events in the same roll, so they are dependent. If we look at the probabilistic definition, indeed we see that for $$\; A_i, A_j, i \neq j \;$$, then $$\; P(A_i \cap A_j) = 0 \neq P(A_i) P(A_j) = \frac{1}{36}\;$$, which shows that they are not independent.
• If we look at different die rolls, then what is the probability of obtaining first a 1 and then a 3? In this case, the events are independent by definition, so rather than asking the question of whether they are dependent or independent, we are interested in computing probabilities assuming they are independent. This probability is $$\; P(1) P(3) = \frac{1}{36} \;$$. But note that here we are asking a different question.
• Thanks a lot @mgarort for the extremely clear description! You've helped me understand this notation, and clarified that there are indeed two different questions of "independence" here. As whuber said, I had realised that there were no sequential events in my question. However in the book, Wasserman discusses only sequential events in this section (both before and after the snippet I posted), which made me think that it must apply in a way that I wasn't understanding. IIUC now, the formula is not at all relevant to sequential events, and it's just a slightly confusing layout in the book... Feb 7, 2021 at 17:52
• Happy it helped! :) These probability problems are not difficult per se, but rather a bit tricky in the notation at first... Once you are used to the concepts of "sample space", "outcome", "event", etc, and realize that they have very specific definitions, everything becomes clearer. Feb 7, 2021 at 18:21