I am studying probability theory from Wasserman's All of Statistics. The author does not mention the concept of a probability space, although he does mention all of its components separately. Please let me know if my understanding of the below ideas is correct.

Suppose we have two discrete random variables $X$ and $Y$. $X$ corresponds to $\Omega_X$ in the probability space $(\Omega_X, \mathcal{F}_X, P_X)$, and $Y$ corresponds to $\Omega_Y$ in the probability space $(\Omega_Y, \mathcal{F}_Y, P_Y)$.

If we consider ($X, Y$) as a pair, is it true that they correspond to a sample space $\Omega_{XY}$ in the probability space $(\Omega_{XY}, \mathcal{F}_{XY}, P_{XY})$, where $P_{XY}$ is the joint mass function $P(X = x, Y = y)$?

  • 1
    $\begingroup$ What exactly do you mean by "$\Omega_{XY}$"? You might find my account of random variables at stats.stackexchange.com/a/96000/919 to be helpful in clarifying that. $\endgroup$
    – whuber
    Aug 30 '18 at 18:31
  • $\begingroup$ Suppose $\Omega_X = \{1, 2\}$ and $\Omega_Y = \{3, 4\}$. Then $\Omega_{XY} = \{(1, 3), (1, 4), (2, 3), (2, 4)\}$. Is this view incorrect? I'm still thinking about your post, because there, $X$ and $Y$ are both associated with a single sample space $\Omega$. $\endgroup$
    – tmakino
    Aug 30 '18 at 20:24
  • 2
    $\begingroup$ That's a great way to view it. You're describing part of the construction of a product of probability spaces in which the underlying set $\Omega_{XY}$ is the Cartesian product of the sets and the sigma algebra is the product sigma algebra. Note that a discrete random variable is not necessarily defined on a finite sample space. For instance, if $X$ has a Normal distribution it cannot possibly be defined on a finite sample space. Yet, the indicator of the event $X \gt 0,$ say, is a discrete variable that is a fortiori defined on a non-finite sample space. $\endgroup$
    – whuber
    Aug 30 '18 at 20:31

What you write is true in some ways, though I don't like your choice of notation which can hide a multitude of traps for the unwary. The product sample space $\Omega_{X,Y}$ (note the comma between $X$ and $Y$) is indeed the Cartesian product of $\Omega_{X}$ and $\Omega_{Y}$, that is, for example, if $\Omega_{X} = \{a, b\}$ and $\Omega_{Y} = \{c, d\}$, then $\Omega_{X,Y} = \Omega_X\times \Omega_Y$ is a set of 4 ordered pairs of real numbers, specifically $$\Omega_{X,Y} = \Omega_X\times \Omega_Y = \big\{(a, c),\, (a,d),\, (b,c),\, (b,d)\big\}.$$ Now, the non-trivial sigma algebras $\mathcal F_{X}$ and $\mathcal F_{Y}$ defined on $\Omega_{X}$ and $\Omega_{Y}$ are their respective power sets \begin{align} \mathcal F_{X} &= 2^{\Omega_X} = \big\{\emptyset,\, \{a\},\, \{b\},\, \Omega_X\big\}\\ \mathcal F_{Y} &= 2^{\Omega_Y} =\big\{\emptyset,\, \{c\},\, \{d\},\, \Omega_Y\big\} \end{align} but the product sigma algebra $\mathcal F_{X,Y}$ is not $\mathcal F_{X}\times \mathcal F_Y$, that is, $$\mathcal F_{X,Y} = 2^{\Omega_{X,Y}} \neq 2^{\Omega_X}\times 2^{\Omega_Y} = \mathcal F_X\times \mathcal F_Y.$$ Note, for example, that $2^{\Omega_{X,Y}}$ contains the event $\big\{(a,c),\, (b,d)\big\}$ since this set is a subset of the product sample space $\Omega_{X,Y}$ whereas $2^{\Omega_X}\times 2^{\Omega_X} = \mathcal F_X\times \mathcal F_Y$ does not.

Turning to probabilities, it is an "obvious" choice to set the probability of the singleton event $\{(x,y)\}$ (here $x \in \Omega_X, y \in \Omega_Y$) to $P(\{x\})P(\{y\})$ which effectively assumes physical independence (implying statistical independence) of the two experiments, but there is no need for such an assumption since the only requirement is that the joint probability measure $P_{X,Y}$ have marginal measures $P_X$ and $P_Y$ respectively. If, say, $P_X(\{a\}) = P_Y(\{c\}) = p$,then the assignments $$\begin{array}{ll} P_{X,Y}(\{(a,c)\}) = p^2 & P_{X,Y}(\{(a,d)\}) = p(1-p)\\ P_{X,Y}(\{(b,c)\}) = p(1-p) & P_{X,Y}(\{(b,d)\}) = (1-p)^2\\ \\ P_{X,Y}^\prime(\{(a,c)\}) = p & P_{X,Y}^\prime(\{(a,d)\}) = 0 \\ P_{X,Y}^\prime(\{(b,c)\}) = 0 & P_{X,Y}^\prime(\{(b,d)\}) = 1-p \end{array}$$ both correspond to the same marginal distribution $$P_X(\{a\}) = P_Y(\{c\}) = p, ~P_X(\{b\}) = P_Y(\{d\}) = 1-p.$$

In short, it is not enough to glibly write

they correspond to a sample space $\Omega_{XY}$ in the probability space $(\Omega_{XY}, \mathcal{F}_{XY}, P_{XY})$

without thinking a little bit as to what exactly is meant by $\mathcal{F}_{XY}$ and $P_{XY}$, or, in more careful notation, $\mathcal F_{X,Y}$ and $P_{X,Y}$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.