How do you get the double sum or integral from $E(X+Y)$ (expected value)?

Question

I was given a proof for $E(X+Y)$ = $E(X)+E(Y)$ for cases where both variables are either discrete or continuous:

Discrete:

$$ \begin{align*} E(X+Y) &=\sum_{x\in\mathcal X}\sum_{y\in\mathcal Y}(x+y)p_{X,Y}(X=x, Y=y)\\ &= \cdots \end{align*}$$

Continuous:

$$ \begin{align*} E(X+Y) &=\int_\mathcal X\int_\mathcal Y(x+y)f_{X, Y}(x, x)dxdy\\ &=\cdots \end{align*} $$

I am not sure how they get double sums or double integrals from the definition of $E(X)$ with a single sum or integral. What is the intuition behind it or is there some mathematical logic behind it?

I know that for one discrete random variable $X$ where $x_1, x_2, \cdots$ are the values of $X$ and $p_X(x)$ is the probability mass function of $X$:

$$E(X)=\sum_{x\in\mathcal X}x_ip_X(x_i)$$

and likewise for one continuous random variable, with $f_X(x)$ being the probability density function for $X$:

$$E(X)=\int_{\mathcal X}x_if_X(x_i)\,\text{d}x_i$$

I am not sure how to get the double sums or integrals from the definitions I have given.

Answer 1

If you assume a joint density $p(x,y)$ for $X$ and $Y$ (I have not said anything about the dependency between $X$ and $Y$ here, they can be correlated). The definition of $E(X+Y)$ is:

$$E(X + Y) = \int_\mathcal Y\int_\mathcal X(x+y)p(x,y)dxdy$$ Further, using the fact that the order of integration can be interchanged and that the integral is a linear operation:

\begin{align} \int_\mathcal Y\int_\mathcal X(x+y)p(x,y)dxdy &= \int_\mathcal Y\int_\mathcal X \left(xp(x,y) + yp(x,y)\right) dx dy = \\ \int_\mathcal X x \int_\mathcal Y p(x,y)dydx + \int_\mathcal Y y \int_\mathcal X p(x,y)dxdy &= \int_\mathcal X xp(x) dx + \int_\mathcal Y yp(y)dy \end{align}

since $\int_\mathcal X p(x,y) dx = p(y)$ and $\int_\mathcal Y p(x,y) dy = p(x)$. Now we simply have the expectation of $X$ and $Y$ separately and we know that

$$\int_\mathcal X xp(x) dx = E(X)$$ and $$\int_\mathcal Y yp(y) dy = E(Y)$$ and therefore $$E(X+Y) = E(X) + E(Y).$$

EDIT

If $X$ and $Y$ are independent we have $p(x,y) = p(x)p(y)$. Therefore the last line of the proof simplifies

$$\int_\mathcal X xp(x)\left(\int_\mathcal Y p(y)dy\right)dx + \int_\mathcal Y y p(y)\left(\int_\mathcal X p(x)dx\right)dy = \int_\mathcal X xp(x) dx + \int_\mathcal Y yp(y)dy, $$ since $\int_\mathcal Y p(y)dy = 1$ (same for $x$).

Answer 2

The question of double versus simple sum is mostly a matter of mathematical simplicity.

First, as pointed by Stubborn Atom, the question is related with the so-poorly-called "Law of the Unconscious Statistician": If one first defines a random variable $Z$ as $$Z=X+Y$$ with associated pdf or pmf $p_Z$, the expectation of $Z$ is a single sum$$\mathbb E_Z[Z] = \sum_z zp_Z(z)$$or single integral$$\mathbb E_Z[Z] = \int_\mathcal Z zp_Z(z)\text{d}z.$$ If one wants to avoid deriving the distribution of $Z$, $X+Y$ is a particular function of $(X,Y)$, $$\psi(X,Y)=X+Y$$ and its expectation $\mathbb E_{(X,Y)}[\psi(X,Y)]$ is a sum or integral over the domain of variation of the random variable $(X,Y)$, $\mathcal X\times\mathcal Y$, which can be written as a double sum. Note however that the double summation$$\sum_{x\in\mathcal X}\sum_{y\in\mathcal Y}\cdots$$is a matter of convention as it can also be represented as a single summation$$\sum_{(x,y)\in\mathcal X\times\mathcal Y}\cdots$$ Similarly, in measure theory, generic integral symbols like$$\int_\mathcal G f(x)\text{d}x$$are customarily used for integrals over multidimensional sets $\mathcal G\subset\mathbb R$ when $k>1$.