I know that the mean of the sum of independent variables is the sum of the means of each independent variable. Does this apply to dependent variables as well?

  • @feetwet, just removing "thanks" isn't really important enough to bump a thread from 18 months ago. FWIW, I voted to reject this edit (but 2 others approved, so you would not have otherwise seen my comment). – gung May 24 '16 at 17:59
  • @gung - All sorts of stuff can mess with the "Active" question view. Your observation has been made often, and AFAIK the Stack Exchange policy is that, despite that drawback, valid minor edits are a good thing. – feetwet May 24 '16 at 18:13
  • 1
    @feetwet, I'm not sure how relevant a meta.Photography post is here. Each SE site has their own meta, and has their own policies, decided by the community. You might want to look at the relevant meta.CV threads, eg, this one: Handling “suggested edits” to posts. You might note whuber's answer quotes Jeff Atwood, "tiny edit[s], like ...removing only the salutation from a post. ...reject them, with extreme prejudice", and joran makes the point, "My threshold for when an edit is too minor is inversely related to the age of the question". – gung May 24 '16 at 18:20
  • 1
    @gung the Photography post I referenced links to a significant and more recent Meta Stack Exchange Q&A on the subject. But if whuber's 4-year-old answer is still canonical for Cross Validated I'll respect that going forward. – feetwet May 24 '16 at 20:51
up vote 14 down vote accepted

Expectation (taking the mean) is a linear operator.

This means that, amongst other things, $\mathbb{E}(X + Y) = \mathbb{E}(X) + \mathbb{E}(Y)$ for any two random variables $X$ and $Y$ (for which the expectations exist), regardless of whether they are independent or not.

We can generalise (e.g. by induction) so that $\mathbb{E}\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n \mathbb{E}(X_i)$ so long as each expectation $\mathbb{E}(X_i)$ exists.

So yes, the mean of the sum is the same as the sum of the mean even if the variables are dependent. But note that this does not apply for the variance! So while $\mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y)$ for independent variables, or even variables which are independent but uncorrelated, the correct general formula is $\mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) + 2\mathrm{Cov}(X, Y)$ where $\mathrm{Cov}$ is the covariance of the variables.

TL; DR:
Assuming it exists, the mean is an expected value, and the expected value is an integral, and the integrals have the linearity property with respect to sums.

TS; DR:
Since we are dealing with the sum of random variables $Y_n = \sum_{i=1}^n X_i$, i.e. of a function of many of them, the mean of the sum $E(Y_n)$ is with respect to their joint distribution (we assume that all means exist and are finite) Denoting $\mathbf X$ the multivariate vector of the $n$ r.v.'s, their joint density can be written as $f_{\mathbf X}(\mathbf x)= f_{X_1,...,X_n}(x_1,...,x_n)$ and their joint support $D = S_{X_1} \times ...\times S_{X_n}$ Using the Law of Unconcscious Statistician we have the multiple integral

$$E[Y_n] = \int_D Y_nf_{\mathbf X}(\mathbf x)d\mathbf x$$.

Under some regularity conditions we can decompose the multiple integral into an $n$-iterative integral:

$$E[Y_n] = \int_{S_{X_n}}...\int_{S_{X_1}}\Big[\sum_{i=1}^n X_i\Big]f_{X_1,...,X_n}(x_1,...,x_n)dx_1...dx_n $$

and using the linearity of integrals we can decompose into

$$ = \int_{S_{X_n}}...\int_{S_{X_1}}x_1f_{X_1,...,X_n}(x_1,...,x_n)dx_1...dx_n \; + ...\\ ...+\int_{S_{X_n}}...\int_{S_{X_1}}x_nf_{X_1,...,X_n}(x_1,...,x_n)dx_1...dx_n $$

For each $n$-iterative integral we can re-arrange the order of integration so that, in each, the outer integration is with respect to the variable that is outside the joint density. Namely,

$$\int_{S_{X_n}}...\int_{S_{X_1}}x_1f_{X_1,...,X_n}(x_1,...,x_n)dx_1...dx_n = \\\int_{S_{X_1}}x_1\int_{S_{X_n}}...\int_{S_{X_2}}f_{X_1,...,X_n}(x_1,...,x_n)dx_2...dx_ndx_1$$

and in general

$$\int_{S_{X_n}}...\int_{S_{X_j}}...\int_{S_{X_1}}x_jf_{X_1,...,X_n}(x_1,...,x_n)dx_1...dx_j...dx_n =$$ $$=\int_{S_{X_j}}x_j\int_{S_{X_n}}...\int_{S_{X_{j-1}}}\int_{S_{X_{j+1}}}...\int_{S_{X_1}}f_{X_1,...,X_n}(x_1,...,x_n)dx_1...dx_{j-1}dx_{j+1}......dx_ndx_j$$

As we calculate one-by-one the integral in each $n$-iterative integral (starting from the inside), we "integrate out" a variable and we obtain in each step the "joint-marginal" distribution of the other variables. Each $n$-iterative integral therefore will end up as $\int_{S_{X_j}}x_jf_{X_j}(x_j)dx_j$.

Bringing it all together we arrive at

$$E[Y_n ] = E[\sum_{i=1}^n X_i] = \int_{S_{X_1}}x_1f_{X_1}(x_1)dx_1 +...+\int_{S_{X_n}}x_nf_{X_n}(x_n)dx_n $$

But now each simple integral is the expected value of each random variable separately, so

$$ E[\sum_{i=1}^n X_i] = E(X_1) + ...+E(X_n) $$ $$= \sum_{i=1}^nE(X_i) $$

Note that we never invoked independence or non-independence of the random variables involved, but we worked solely with their joint distribution.

  • @ssdecontrol This is one upvote I do appreciate, indeed. – Alecos Papadopoulos Dec 24 '14 at 17:21
  • 1
    The expansion into iterated integrals and back again is unnecessary. It complicates a simple argument. You could replace the "TS;DR" section with its last sentence and have a fine answer. – whuber May 24 '16 at 18:18
  • @whuber One and a half years later, it still escapes me (I mean, without using the "linearity of the expectation operator" fact, that has already been used by the other answer). Any hint so I can rework the answer towards this simple argument? – Alecos Papadopoulos May 24 '16 at 18:24
  • I think the argument is superfluous. The key to the whole thing is your observation in the last sentence. – whuber May 24 '16 at 18:45

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.