4
$\begingroup$

I completely understand what is the intuitive sense of the single R.V mean. However, I was solving a math problem to find the expected value of the sum of two independent random variable, which is can be found by

$$\mathbb E[x+y] = \int (x+y) p(x) p(y)\, dx \,dy\tag 1$$

So I said, yes it makes sense, since we want to find the expected value for both of the two random variable. Then I criticized myself by two points

  1. What does both mean here? That two of the event occur? That is the product of the two R.V not the sum! So it makes sense to write an equation for $\mathbb E[x+y] = \int (x+y) p(x) p(y) \,dx\, dy$
  2. If we write the equation $(1)$ as $\mathbb E[x+y] = \int x p(x) p(y)\, dx \,dy + \int y p(x) p(y) \,dx\, dy$. What does each term mean? What is the meaning (if we take the first term) of finding the average of random variable $X$ over the probability of $X$ and $Y$?

I did not find answers to those questions.

$\endgroup$
  • 3
    $\begingroup$ The language used in this question suggests there may be some underlying misunderstanding of what a random variable is, because it seems to be confused with an "event." Please see stats.stackexchange.com/questions/50 for clarification. Concerning (2), the terms are the expectations of $X$ and $Y$, by definition. $\endgroup$ – whuber Jan 28 '17 at 19:37
  • $\begingroup$ @whuber It seems yes, I didn't realize that :) $\endgroup$ – hbak Jan 28 '17 at 20:42
4
$\begingroup$

Recall that for any random variables $X$ and $Y$ with a joint probability distribution function $p(x,y)$, the expected value of $X+Y$ is $$ \mathbb{E}_{X,Y}[X+Y] = \int_x\int_y (x+y) p(x,y) dydx = \int_x x \int_y p(x,y)dydx + \int_y y \int_x p(x,y) dxdy$$

In the special case that $x$ and $y$ are independent, $p(x,y) = p_X(x)p_Y(y)$ and we can write the integral in the form that you had it, where $p_X$ is the marginal probability distribution of $X$ and $p_Y$ the marginal probability distribution of $Y$: $$ \mathbb{E}[X+Y] = \int_x\int_y (x+y) p_X(x)p_Y(y) dydx$$

This is where the "product" is coming from - it represents the (infinitesmal) probability that $X=x$ and $Y=y$, which we use to weight $x+y$ appropriately when computing the expected value of $X+Y$. Think of it as searching over all possible combinations of $X$ and $Y$, and for each combination you are evaluating the value of $X+Y$ and weighting it by the probability the combination occurs.

Since you are already adding two random variables, presumably there is a meaning to the sum of the variables. For example if it is travel time, $X$ could be the time it takes for the first trip, $Y$ for the second, and $X+Y$ is the total duration. $\mathbb{E}[X+Y]$ is the expected time of the total trip.

Linearity of expectation might help out a bit for understanding the last question. Remember that $\mathbb{E}[X+Y] = \mathbb{E}[X] + \mathbb{E}[Y]$. For the independent case, if you simplify the two integrals you'll get $$ \mathbb{E}[X+Y] = \int_X x p_X(x) dx + \int_Y yp_Y(y)dy$$ the first integral is the expected value of $X$, the second term is the expected value of $Y$. In the travel example, you would expected the total trip time to be the expected time of the first trip plus the expected time of the second trip.

$\endgroup$
  • $\begingroup$ Thanks. How would you interpret the E[x.y] based on your example? $\endgroup$ – hbak Jan 28 '17 at 19:02
  • $\begingroup$ You mean the product? I am unsure what you mean by $x.y$. $\endgroup$ – combo Jan 28 '17 at 19:03
  • $\begingroup$ Yes the product. $\endgroup$ – hbak Jan 28 '17 at 19:04
  • 2
    $\begingroup$ This answer starts out correctly but then degenerates into something that seems to suggest that linearity of expectation holds only if $X$ and $Y$ are independent. What is missing is the realization that in the very first displayed formula above, $\int_y p(x,y)dy$ and $\int_x p(x,y)dx$ have values $p_X(x)$ and $p_Y(y)$ respectively and so even when $X$ and $Y$ are not independent, $$E[X+Y]=\int_x xp_X(x)dx + \int_yyp_Y(y)dy=E[X]+E[Y].$$. -1 pending corrections and improvement of notation. It is generally accepted that a symbol should mean the same thing wherever it occurs.... $\endgroup$ – Dilip Sarwate Jan 28 '17 at 20:45
  • 1
    $\begingroup$ (continued)... Try that on $$\int_x xp(x)dx + \int_y yp(y)dy$$ where $p$ stands for two entirely different functions depending on what the argument is instead of the generally accepted notion that $p(x)$ is the value of the function $p(\cdot)$ at $x$ and $p(y)$ the value of the same function at $y$. $\endgroup$ – Dilip Sarwate Jan 28 '17 at 20:48
2
$\begingroup$

If $X$ and $Y$ are random variables (defined on the same probability space: ignore this remark if it confuses you), then we can regard $(X,Y)$ as a random vector (also called a bivariate random variable) and $X$ and $Y$ individually as special kinds of functions of $(X,Y)$ -- called projections or projection maps if you want to use fancy words. Another function of $(X,Y)$ is $X+Y$ (called the "sum" function, of course, what else?) and what this means is that if on a particular trial of the experiment, $X$ and $Y$ took on values $x$ and $y$ respectively (equivalently, $(X,Y)$ had value $(x,y)$), then this sum random variable (denote it by $Z$) has taken on value $x+y$ on this particular trial. There is no notion of "both occurring"; as whuber points out in his comment, you are confusing the concepts of events and random variables.

So, if $W$ is a random variable, what is $E[g(W)]$, the expected value of the function $V = g(W)$ of the random variable $W$? There are two standard ways of finding the answer: if we know, or can determine, the distribution of $V$, then we can use the definition of expectation. For example, if $V$ is a continuous random variable with pdf $f_V(v)$, then $$E[V] = \int_{-\infty}^\infty v\cdot f_V(v) \,\mathrm dv.$$ Alternatvely, we can use the Law of the Unconscious Statistician or LOTUS and determine the value of $E[V]=E[g(W)]$ as $$E[V]=E[g(W)] = \int_{-\infty}^\infty g(w)\cdot f_W(w) \,\mathrm dw$$ where $f_W(w)$ is the pdf of $W$. Now, LOTUS applies to functions of bivariate (and more generally multivariate) random variables also, and we can find $E[Z] =E[X+Y]$ via $$E[Z]=E[X+Y] = \int_{-\infty}^\infty \int_{-\infty}^\infty (x+y)\cdot f_{X,Y}(x,y) \,\mathrm dx \,\mathrm dy\tag{1}$$ where $f_{X,Y}(x,y)$ is the joint pdf of $X$ and $Y$ or just the pdf of the bivariate random variable $(X,Y)$. As a special case, when $X$ and $Y$ are independent random variables, $f_{X,Y}(x,y) = f_{X}(x)f_{Y}(y)$ for all $x$ and $y$, and so we get the formula shown in your question. But it is very important that you understand that $(1)$ always holds, regardless of independence etc (for jointly continuous random variables).

A funny thing happens on the way to the forum as one massages the forumula $(1)$. We have that \begin{align} E[X+Y] &= \int_{-\infty}^\infty \int_{-\infty}^\infty (x+y)\cdot f_{X,Y}(x,y) \,\mathrm dx \,\mathrm dy\\ &= \int_{-\infty}^\infty \int_{-\infty}^\infty x\cdot f_{X,Y}(x,y) \,\mathrm dx \,\mathrm dy + \int_{-\infty}^\infty \int_{-\infty}^\infty y \cdot f_{X,Y}(x,y) \,\mathrm dx \,\mathrm dy\\ &= \int_{-\infty}^\infty x\int_{-\infty}^\infty f_{X,Y}(x,y) \,\mathrm dy \,\mathrm dx + \int_{-\infty}^\infty y \int_{-\infty}^\infty f_{X,Y}(x,y) \,\mathrm dx \,\mathrm dy\\ &= \int_{-\infty}^\infty x\left[\int_{-\infty}^\infty f_{X,Y}(x,y) \,\mathrm dy \right] \,\mathrm dx + \int_{-\infty}^\infty y \left[\int_{-\infty}^\infty f_{X,Y}(x,y) \,\mathrm dx\right] \,\mathrm dy\\ &= \int_{-\infty}^\infty x \cdot f_{X}(x) \,\mathrm dx + \int_{-\infty}^\infty y\cdot f_{Y}(y) \,\mathrm dy\\ E[X+Y] &= E[X] + E[Y]\tag{2} \end{align} The result $(2)$ is a special case of the linearity of expectation because the argument above can be applied to show that $E[aX+bY] = aE[X]+bE[Y]$, that is, expectation behaves like a linear operation with respect to random variables: the expectation of a weighted sum is the weighted sum of the expectations.

Linearity of expectation is a very general result. It holds for all random variables, not just the jointly continuous ones as in the calculation above, or for independent random variables only as the answer by Stefan Jorgenson seems to be suggesting.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.