1
$\begingroup$

I was given a proof for $E(X+Y)$ = $E(X)+E(Y)$ for cases where both variables are either discrete or continuous:

Discrete:

$$ \begin{align*} E(X+Y) &=\sum_{x\in\mathcal X}\sum_{y\in\mathcal Y}(x+y)p_{X,Y}(X=x, Y=y)\\ &= \cdots \end{align*}$$

Continuous:

$$ \begin{align*} E(X+Y) &=\int_\mathcal X\int_\mathcal Y(x+y)f_{X, Y}(x, x)dxdy\\ &=\cdots \end{align*} $$

I am not sure how they get double sums or double integrals from the definition of $E(X)$ with a single sum or integral. What is the intuition behind it or is there some mathematical logic behind it?

I know that for one discrete random variable $X$ where $x_1, x_2, \cdots$ are the values of $X$ and $p_X(x)$ is the probability mass function of $X$:

$$E(X)=\sum_{x\in\mathcal X}x_ip_X(x_i)$$

and likewise for one continuous random variable, with $f_X(x)$ being the probability density function for $X$:

$$E(X)=\int_{\mathcal X}x_if_X(x_i)\,\text{d}x_i$$

I am not sure how to get the double sums or integrals from the definitions I have given.

$\endgroup$
  • 2
    $\begingroup$ Have a look at en.m.wikipedia.org/wiki/Law_of_the_unconscious_statistician. $\endgroup$ – StubbornAtom Aug 7 '20 at 6:35
  • $\begingroup$ The take away point is that the expectation notation is highly efficient and synthetic. The same symbol $\mathbb{E}$ can mean (ah ha) different things depending on its argument, it is usually clear from context what it means, but is a source of confusion for many students. $\endgroup$ – Three Diag Aug 7 '20 at 8:35
1
$\begingroup$

If you assume a joint density $p(x,y)$ for $X$ and $Y$ (I have not said anything about the dependency between $X$ and $Y$ here, they can be correlated). The definition of $E(X+Y)$ is:

$$E(X + Y) = \int_\mathcal Y\int_\mathcal X(x+y)p(x,y)dxdy$$ Further, using the fact that the order of integration can be interchanged and that the integral is a linear operation:

\begin{align} \int_\mathcal Y\int_\mathcal X(x+y)p(x,y)dxdy &= \int_\mathcal Y\int_\mathcal X xp(x,y) + yp(x,y) dx dy = \\ \int_\mathcal X\int_\mathcal Y p(x,y)dydx + \int_\mathcal Y y \int_\mathcal X p(x,y)dxdy &= \int_\mathcal X xp(x) dx + \int_\mathcal Y yp(y)dy \end{align}

since $\int_\mathcal X p(x,y) dx = p(y)$ and $\int_\mathcal Y p(x,y) dy = p(x)$. Now we simply have the expectation of $X$ and $Y$ separately and we know that

$$\int_\mathcal X xp(x) dx = E(X)$$ and $$\int_\mathcal Y yp(y) dy = E(Y)$$ and therefore $$E(X+Y) = E(X) + E(Y).$$

EDIT

If $X$ and $Y$ are independent we have $p(x,y) = p(x)p(y)$. Therefore the last line of the proof simplifies

$$\int_\mathcal X xp(x)(\int_\mathcal Y p(y)dy)dx + \int_\mathcal Y y p(y)(\int_\mathcal X p(x)dx)dy = \int_\mathcal X xp(x) dx + \int_\mathcal Y yp(y)dy, $$ since $\int_\mathcal Y p(y)dy = 1$ (same for $x$).

$\endgroup$
  • $\begingroup$ How can you assume they are joint random variables? $\endgroup$ – user12055579 Aug 7 '20 at 6:24
  • 1
    $\begingroup$ By the way, thank you for the answer! $\endgroup$ – user12055579 Aug 7 '20 at 7:16
  • 2
    $\begingroup$ The brackets are just to emphasize that you can solve the inner integral first and then solve the outer one. $\endgroup$ – J.C.Wahl Aug 7 '20 at 7:20
  • 3
    $\begingroup$ @user12055579 If $X$ and $Y$ are two random variables, you can always ask about the probability that $X < a$ and $Y < b$ simultaneously. We can define a function $F(a, b) = P(X < a \text{ and } Y < b)$ to be this probability, and we call it the joint cdf (cumulative distribution function). We don't have to make any assumptions on $X$ and $Y$ to do this. This is what is meant when someone says that $X$ and $Y$ are "jointly distributed": it's not an assumption on the random variables, simply a way of characterizing them with a function that we can construct from them. $\endgroup$ – Eric Perkerson Aug 7 '20 at 8:50
  • 1
    $\begingroup$ No, I think you still misunderstand. In a single context (probability space), there's no such thing as two random variables that aren't jointly distributed like you seem to be implying. Every set of random variables on a probability space are jointly distributed with a cdf defined as above. $\endgroup$ – Eric Perkerson Aug 7 '20 at 8:56
2
$\begingroup$

The question of double versus simple sum is mostly a matter of mathematical simplicity.

First, as pointed by Stubborn Atom, the question is related with the so-poorly-called "Law of the Unconscious Statistician": If one first defines a random variable $Z$ as $$Z=X+Y$$ with associated pdf or pmf $p_Z$, the expectation of $Z$ is a single sum$$\mathbb E_Z[Z] = \sum_z zp_Z(z)$$or single integral$$\mathbb E_Z[Z] = \int_\mathcal Z zp_Z(z)\text{d}z.$$ If one wants to avoid deriving the distribution of $Z$, $X+Y$ is a particular function of $(X,Y)$, $$\psi(X,Y)=X+Y$$ and its expectation $\mathbb E_{(X,Y)}[\psi(X,Y)]$ is a sum or integral over the domain of variation of the random variable $(X,Y)$, $\mathcal X\times\mathcal Y$, which can be written as a double sum. Note however that the double summation$$\sum_{x\in\mathcal X}\sum_{y\in\mathcal Y}\cdots$$is a matter of convention as it can also be represented as a single summation$$\sum_{(x,y)\in\mathcal X\times\mathcal Y}\cdots$$ Similarly, in measure theory, generic integral symbols like$$\int_\mathcal G f(x)\text{d}x$$are customarily used for integrals over multidimensional sets $\mathcal G\subset\mathbb R$ when $k>1$.

$\endgroup$
  • $\begingroup$ What is $\psi$ and what is variation? $\endgroup$ – user12055579 Aug 7 '20 at 8:39
  • $\begingroup$ And what do you replace the $\cdots$ with? $\endgroup$ – user12055579 Aug 7 '20 at 8:40
  • $\begingroup$ Wait nevermind, I get it. Variation would be $\mathcal X\times\mathcal Y$ and the $\cdots$ would be replaced with the stuff in the question. Correct me if I am wrong. Thanks! $\endgroup$ – user12055579 Aug 7 '20 at 8:47
  • $\begingroup$ $\psi$ is the function defined in the answer: $\psi(x,y)=x+y$ and the domain of variation of $(X,Y)$ is the set of all possible values taken by the random pair, sometimes called the support of the random vector. As for $\cdots$ it can be replaced by $\psi(x,y)p_{(X,Y)}(x,y)$ or $\psi(x,y)p_{(X,Y)}(x,y)\text{d}(x,y)$ $\endgroup$ – Xi'an Aug 7 '20 at 8:57
  • $\begingroup$ Okay, I see, thanks. I have a question which is why is it that: $\psi(X,Y)=X+Y = \psi(x,y)=x+y$? $\endgroup$ – user12055579 Aug 7 '20 at 9:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.