Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
suppose we have two discrete random variables:
$X: \{$6 sided dice rolls$\}$ $\rightarrow \{1..6\}$ (following uniform distribution)
$Y: \{$coin flips$\}$ $\rightarrow \{0,1\}$ (following uniform distribution)
How can we add X+Y=Z, for Z a random variable? It makes no sense to me since X and Y have different sample space domains, but shouldn't there be a way to do this?
It's a good question, interpreted in the following way: the random variable $X$ determined by the coin experiment has a sample space of $\Omega_1=\{\text{Heads},\ \text{Tails}\}$ while the random variable $Y$ determined by the roll of a die has a sample space $\Omega_2$ consisting of the six possible stable orientations of the die on the table. These obviously are not the same sample space, so what sense does it make to add them?
I will explain one approach to this problem in two ways: first using mathematical terminology and then again using a standard physical model (or metaphor) for probability spaces based on random sampling.
Implicitly, we form the product space $\Omega=\Omega_1\times\Omega_2.$ Its elements consist of all the ordered pairs $(\omega_1,\omega_2)$ where $\omega_1\in\Omega_1$ and $\omega_2\in\Omega_2.$ The original random variables define new random variables on $\Omega:$ $X$ defines the random variable
$$(\omega_1, \omega_2) \to X(\omega_1)$$
while $Y$ defines the random variable
$$(\omega_1,\omega_2) \to Y(\omega_2).$$
It is a traditional abuse of notation to reuse the symbols $X$ and $Y$ for these new functions on this new space $\Omega.$ It is also understood, when $X$ or $Y$ have continuous distributions, that $\Omega$ is given a sufficiently rich set of events (its sigma algebra) to make these new functions into bona fide random variables: that is, they will be measurable.
Now it makes sense to write "$X+Y$" because it can be defined by pointwise addition (as usual) via
$$(X+Y)(\omega_1,\omega_2) = X(\omega_1,\omega_2) + Y(\omega_1,\omega_2) = X(\omega_1) + Y(\omega_2)$$
where the last equality comes from the definitions of $X$ and $Y$ on $\Omega.$
If we continue to insist that the two original sample spaces were different, there is no way to model any dependency among them. Each has its probability function $\mathbb{P}_1$ and $\mathbb{P}_2.$ One thing we can always do, though, is to give the events in $\Omega$ the probabilities $\mathbb P$ they must have according to the definition of independence. In particular, when $E_1\subset \Omega_1$ and $E_2\subset\Omega_2$ are events, then $E_1\times E_2 = \{(\omega_1,\omega_2)\in\Omega\mid \omega_1\in E_1\text{ and } \omega_2\in E_2\}$ is an event and
$$\mathbb{P}(E_1\times E_2) = \mathbb{P}_1(E_1)\,\mathbb{P}_2(E_2).$$
This enables us to do probability calculations with random variables defined on $\Omega$ including, for example, $X+Y.$
In general, though, people usually assume this product construction has already been carried out, so that in effect $X$ and $Y$ were defined on a common sample space all along. This permits us to model arbitrary dependencies among these variables, simply by defining any valid sigma algebra and probability function we like on $\Omega.$
In terms of the tickets in a box metaphor, $\Omega_1$ is a box with two tickets on it (one for each side of the coin) and $\Omega_2$ is a box with six tickets on it (one for each side of the die). $X$ consists of writing $0$ on one of the coin tickets and $1$ on the other coin ticket. $Y$ consists of writing the numbers $1,\ldots,6$ on the six die tickets, one number per ticket. The "product box" $\Omega$ is created by tabulating all combinations a ticket from one box coupled with a ticket from the other box, as in this figure where elements of $\Omega_1$ index the columns and elements of $\Omega_2$ index the rows.
I have identified the six sides of the die by giving it a coordinate system in which the die is the cube bounded by the eight points with coordinates $\pm 1,$ so that unit vectors pointing out from its six faces determine those faces, and for brevity I write the vectors without parentheses or commas:
$$\array{& \text{Tails} & \text{Heads} \\ \hline 100: & (\text{Tails}, 100) & (\text{Heads}, 100)\\ 010: & (\text{Tails}, 010) & (\text{Heads}, 010) \\ 001: & (\text{Tails}, 001) & (\text{Heads}, 001) \\ -001: & (\text{Tails}, -001) & (\text{Heads}, -001) \\ -010: & (\text{Tails}, -010) & (\text{Heads}, -010) \\ -100: & (\text{Tails}, -100) & (\text{Heads}, -100) }$$
We cut out the $2\times 6 = 12$ cells of this table and put them into a new box: that's $\Omega.$ These tickets represent all possible combinations of a flip of the coin and a roll of the die.
(Clearly it doesn't matter whether you work with $\Omega_1\times \Omega_2$ or $\Omega_2\times\Omega_1;$ the difference--although a real one at a basic mathematical level as well as in the typographical formatting of the table--is just a matter of notation.)
One way to model the probability on the product space is to go through this process with the actual tickets in the boxes rather than the unique types of tickets as shown above. For instance, if $\Omega_2$ has two tickets for "Tails" and one ticket for "Heads" (modeling a coin that favors Tails 2:1), then the table would have three columns: one for each ticket. When $\Omega_1$ has $m$ tickets and $\Omega_2$ has $n$ tickets, this will create $mn$ new "product" tickets to put into $\Omega.$
If instead we put arbitrary numbers of each product ticket into $\Omega,$ we can change the probabilities of events and create dependencies among the random variables. Thus, in a setting where we wish to add two random variables representing different kinds of outcomes, we usually assume this table has been created and the tickets consist of various cells from the table in various proportions.
Surely they can have different sample spaces (or supports) but you can add them and generate another RV.
If you assume independence between the two, the resulting mass function can be calculated by simply convolving the two. But, a more primitive solution is to list all possible scenarios:
$$\begin{align}&P(Z=1)=P(Y=0)P(X=1)=1/12\\&P(Z=2)=P(Y=0)P(X=2)+P(Y=1)P(X=1)=2/12\\&...\end{align}$$
which will make up the mass function of $Z$.
Remember that X and Y will be the outcome of the roll of a dice/ flip of a coin. So if I understand your question correctly, you will ad the value you get from your first die roll (say 2) to the value of the coin toss (say 0). Resulting in a Z of 2 for example.
Any two sample space can be combined by the operation of direct sum. That's generally denoted by the symbol $\oplus$, although the simple addition symbol is sometimes used when it's considered obvious that the direct sum is intended.
However, in this case, it appears that regular addition is meant. The coin flips are being represented by the integers {0, 1}, so those numbers can be added to the die roll. While the sample space ranges, in the sense of the possible values, are different, in both cases the overall space is the set of integers, so addition is possible.
