# How do you get the double sum or integral from $E(X+Y)$ (expected value)?

I was given a proof for $$E(X+Y)$$ = $$E(X)+E(Y)$$ for cases where both variables are either discrete or continuous:

Discrete:

\begin{align*} E(X+Y) &=\sum_{x\in\mathcal X}\sum_{y\in\mathcal Y}(x+y)p_{X,Y}(X=x, Y=y)\\ &= \cdots \end{align*}

Continuous:

\begin{align*} E(X+Y) &=\int_\mathcal X\int_\mathcal Y(x+y)f_{X, Y}(x, x)dxdy\\ &=\cdots \end{align*}

I am not sure how they get double sums or double integrals from the definition of $$E(X)$$ with a single sum or integral. What is the intuition behind it or is there some mathematical logic behind it?

I know that for one discrete random variable $$X$$ where $$x_1, x_2, \cdots$$ are the values of $$X$$ and $$p_X(x)$$ is the probability mass function of $$X$$:

$$E(X)=\sum_{x\in\mathcal X}x_ip_X(x_i)$$

and likewise for one continuous random variable, with $$f_X(x)$$ being the probability density function for $$X$$:

$$E(X)=\int_{\mathcal X}x_if_X(x_i)\,\text{d}x_i$$

I am not sure how to get the double sums or integrals from the definitions I have given.

• Have a look at en.m.wikipedia.org/wiki/Law_of_the_unconscious_statistician. – StubbornAtom Aug 7 '20 at 6:35
• The take away point is that the expectation notation is highly efficient and synthetic. The same symbol $\mathbb{E}$ can mean (ah ha) different things depending on its argument, it is usually clear from context what it means, but is a source of confusion for many students. – Three Diag Aug 7 '20 at 8:35

If you assume a joint density $$p(x,y)$$ for $$X$$ and $$Y$$ (I have not said anything about the dependency between $$X$$ and $$Y$$ here, they can be correlated). The definition of $$E(X+Y)$$ is:

$$E(X + Y) = \int_\mathcal Y\int_\mathcal X(x+y)p(x,y)dxdy$$ Further, using the fact that the order of integration can be interchanged and that the integral is a linear operation:

\begin{align} \int_\mathcal Y\int_\mathcal X(x+y)p(x,y)dxdy &= \int_\mathcal Y\int_\mathcal X xp(x,y) + yp(x,y) dx dy = \\ \int_\mathcal X\int_\mathcal Y p(x,y)dydx + \int_\mathcal Y y \int_\mathcal X p(x,y)dxdy &= \int_\mathcal X xp(x) dx + \int_\mathcal Y yp(y)dy \end{align}

since $$\int_\mathcal X p(x,y) dx = p(y)$$ and $$\int_\mathcal Y p(x,y) dy = p(x)$$. Now we simply have the expectation of $$X$$ and $$Y$$ separately and we know that

$$\int_\mathcal X xp(x) dx = E(X)$$ and $$\int_\mathcal Y yp(y) dy = E(Y)$$ and therefore $$E(X+Y) = E(X) + E(Y).$$

EDIT

If $$X$$ and $$Y$$ are independent we have $$p(x,y) = p(x)p(y)$$. Therefore the last line of the proof simplifies

$$\int_\mathcal X xp(x)(\int_\mathcal Y p(y)dy)dx + \int_\mathcal Y y p(y)(\int_\mathcal X p(x)dx)dy = \int_\mathcal X xp(x) dx + \int_\mathcal Y yp(y)dy,$$ since $$\int_\mathcal Y p(y)dy = 1$$ (same for $$x$$).

• How can you assume they are joint random variables? – user12055579 Aug 7 '20 at 6:24
• By the way, thank you for the answer! – user12055579 Aug 7 '20 at 7:16
• The brackets are just to emphasize that you can solve the inner integral first and then solve the outer one. – J.C.Wahl Aug 7 '20 at 7:20
• @user12055579 If $X$ and $Y$ are two random variables, you can always ask about the probability that $X < a$ and $Y < b$ simultaneously. We can define a function $F(a, b) = P(X < a \text{ and } Y < b)$ to be this probability, and we call it the joint cdf (cumulative distribution function). We don't have to make any assumptions on $X$ and $Y$ to do this. This is what is meant when someone says that $X$ and $Y$ are "jointly distributed": it's not an assumption on the random variables, simply a way of characterizing them with a function that we can construct from them. – Eric Perkerson Aug 7 '20 at 8:50
• No, I think you still misunderstand. In a single context (probability space), there's no such thing as two random variables that aren't jointly distributed like you seem to be implying. Every set of random variables on a probability space are jointly distributed with a cdf defined as above. – Eric Perkerson Aug 7 '20 at 8:56

The question of double versus simple sum is mostly a matter of mathematical simplicity.

First, as pointed by Stubborn Atom, the question is related with the so-poorly-called "Law of the Unconscious Statistician": If one first defines a random variable $$Z$$ as $$Z=X+Y$$ with associated pdf or pmf $$p_Z$$, the expectation of $$Z$$ is a single sum$$\mathbb E_Z[Z] = \sum_z zp_Z(z)$$or single integral$$\mathbb E_Z[Z] = \int_\mathcal Z zp_Z(z)\text{d}z.$$ If one wants to avoid deriving the distribution of $$Z$$, $$X+Y$$ is a particular function of $$(X,Y)$$, $$\psi(X,Y)=X+Y$$ and its expectation $$\mathbb E_{(X,Y)}[\psi(X,Y)]$$ is a sum or integral over the domain of variation of the random variable $$(X,Y)$$, $$\mathcal X\times\mathcal Y$$, which can be written as a double sum. Note however that the double summation$$\sum_{x\in\mathcal X}\sum_{y\in\mathcal Y}\cdots$$is a matter of convention as it can also be represented as a single summation$$\sum_{(x,y)\in\mathcal X\times\mathcal Y}\cdots$$ Similarly, in measure theory, generic integral symbols like$$\int_\mathcal G f(x)\text{d}x$$are customarily used for integrals over multidimensional sets $$\mathcal G\subset\mathbb R$$ when $$k>1$$.

• What is $\psi$ and what is variation? – user12055579 Aug 7 '20 at 8:39
• And what do you replace the $\cdots$ with? – user12055579 Aug 7 '20 at 8:40
• Wait nevermind, I get it. Variation would be $\mathcal X\times\mathcal Y$ and the $\cdots$ would be replaced with the stuff in the question. Correct me if I am wrong. Thanks! – user12055579 Aug 7 '20 at 8:47
• $\psi$ is the function defined in the answer: $\psi(x,y)=x+y$ and the domain of variation of $(X,Y)$ is the set of all possible values taken by the random pair, sometimes called the support of the random vector. As for $\cdots$ it can be replaced by $\psi(x,y)p_{(X,Y)}(x,y)$ or $\psi(x,y)p_{(X,Y)}(x,y)\text{d}(x,y)$ – Xi'an Aug 7 '20 at 8:57
• Okay, I see, thanks. I have a question which is why is it that: $\psi(X,Y)=X+Y = \psi(x,y)=x+y$? – user12055579 Aug 7 '20 at 9:04