Does the variance of a sum equal the sum of the variances?

Question

Is it (always) true that $$\mathrm{Var}\left(\sum\limits_{i=1}^m{X_i}\right) = \sum\limits_{i=1}^m{\mathrm{Var}(X_i)} \>?$$

Answer 1

The answer to your question is "Sometimes, but not in general".

To see this let $X_1, ..., X_n$ be random variables (with finite variances). Then,

$$ {\rm var} \left( \sum_{i=1}^{n} X_i \right) = E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 \right) - \left[ E\left( \sum_{i=1}^{n} X_i \right) \right]^2$$

Now note that $(\sum_{i=1}^{n} a_i)^2 = \sum_{i=1}^{n} \sum_{j=1}^{n} a_i a_j $, which is clear if you think about what you're doing when you calculate $(a_1+...+a_n) \cdot (a_1+...+a_n)$ by hand. Therefore,

$$ E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 \right) = E \left( \sum_{i=1}^{n} \sum_{j=1}^{n} X_i X_j \right) = \sum_{i=1}^{n} \sum_{j=1}^{n} E(X_i X_j) $$

similarly,

$$ \left[ E\left( \sum_{i=1}^{n} X_i \right) \right]^2 = \left[ \sum_{i=1}^{n} E(X_i) \right]^2 = \sum_{i=1}^{n} \sum_{j=1}^{n} E(X_i) E(X_j)$$

so

$$ {\rm var} \left( \sum_{i=1}^{n} X_i \right) = \sum_{i=1}^{n} \sum_{j=1}^{n} \big( E(X_i X_j)-E(X_i) E(X_j) \big) = \sum_{i=1}^{n} \sum_{j=1}^{n} {\rm cov}(X_i, X_j)$$

by the definition of covariance.

Now regarding Does the variance of a sum equal the sum of the variances?:

• If the variables are uncorrelated, yes: that is, ${\rm cov}(X_i,X_j)=0$ for $i\neq j$, then $$ {\rm var} \left( \sum_{i=1}^{n} X_i \right) = \sum_{i=1}^{n} \sum_{j=1}^{n} {\rm cov}(X_i, X_j) = \sum_{i=1}^{n} {\rm cov}(X_i, X_i) = \sum_{i=1}^{n} {\rm var}(X_i) $$

• If the variables are correlated, no, not in general: For example, suppose $X_1, X_2$ are two random variables each with variance $\sigma^2$ and ${\rm cov}(X_1,X_2)=\rho$ where $0 < \rho <\sigma^2$. Then ${\rm var}(X_1 + X_2) = 2(\sigma^2 + \rho) \neq 2\sigma^2$, so the identity fails.

• but it is possible for certain examples: Suppose $X_1, X_2, X_3$ have covariance matrix $$ \left( \begin{array}{ccc} 1 & 0.4 &-0.6 \\ 0.4 & 1 & 0.2 \\ -0.6 & 0.2 & 1 \\ \end{array} \right) $$ then ${\rm var}(X_1+X_2+X_3) = 3 = {\rm var}(X_1) + {\rm var}(X_2) + {\rm var}(X_3)$

Therefore if the variables are uncorrelated then the variance of the sum is the sum of the variances, but converse is not true in general.

Answer 2

$$\text{Var}\bigg(\sum_{i=1}^m X_i\bigg) = \sum_{i=1}^m \text{Var}(X_i) + 2\sum_{i\lt j} \text{Cov}(X_i,X_j).$$

So, if the covariances average to $0$, which would be a consequence if the variables are pairwise uncorrelated or if they are independent, then the variance of the sum is the sum of the variances.

An example where this is not true: Let $\text{Var}(X_1)=1$. Let $X_2 = X_1$. Then $\text{Var}(X_1 + X_2) = \text{Var}(2X_1)=4$.

Answer 3

I just wanted to add a more succinct version of the proof given by Macro, so it's easier to see what's going on. $\newcommand{\Cov}{\text{Cov}}\newcommand{\Var}{\text{Var}}$

Notice that since $\Var(X) = \Cov(X,X)$

For any two random variables $X,Y$ we have:

\begin{align} \Var(X+Y) &= \Cov(X+Y,X+Y) \\ &= E((X+Y)^2)-E(X+Y)E(X+Y) \\ &\text{by expanding,} \\ &= E(X^2) - (E(X))^2 + E(Y^2) - (E(Y))^2 + 2(E(XY) - E(X)E(Y)) \\ &= \Var(X) + \Var(Y) + 2(E(XY)) - E(X)E(Y)) \\ \end{align} Therefore in general, the variance of the sum of two random variables is not the sum of the variances. However, if $X,Y$ are independent, then $E(XY) = E(X)E(Y)$, and we have $\Var(X+Y) = \Var(X) + \Var(Y)$.

Notice that we can produce the result for the sum of $n$ random variables by a simple induction.

Answer 4

Yes, if each pair of the $X_i$'s are uncorrelated, this is true.

See the explanation on Wikipedia

Answer 5

I just want to add some steps in the very first equivalence of Macro's answer. Indeed I think it can be helpful to retrieve it directly from the usual definition of variance ${\rm var}(X)=E([X-E(X)]^2)$ from which in this particular case we should have:

$$ {\rm var} \left( \sum_{i=1}^{n} X_i \right) = E \left( \left[ \sum_{i=1}^{n} X_i - E\left( \sum_{i=1}^{n} X_i \right)\right]^2 \right) $$

Indeed expanding the argument of the expected value:

$$ E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 -2 \left( \sum_{i=1}^{n} X_i \right) E\left( \sum_{i=1}^{n} X_i \right) + \left[ E\left( \sum_{i=1}^{n} X_i \right) \right]^2 \right) $$

Exploiting the linearity of the expectations, the expected value of a linear combination of random variables is the linear combination of the expected values of the corresponding random variables, such that:

$$ E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 \right) -2 E \left( \left( \sum_{i=1}^{n} X_i \right) E \left( \sum_{i=1}^{n} X_i \right) \right) + E \left( \left[ E\left( \sum_{i=1}^{n} X_i \right) \right]^2 \right) $$

The last term is the expectation of a constant which is equal to the constant itself (by means of the LOTUS theorem). Exploiting again linearity of the expectations in the second term:

$$ E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 \right) -2 E\left( \sum_{i=1}^{n} X_i \right) E \left( \sum_{i=1}^{n} X_i \right) + \left[ E\left( \sum_{i=1}^{n} X_i \right) \right]^2 = $$ $$ = E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 \right) -2 \left[ E \left( \sum_{i=1}^{n} X_i \right) \right] ^2 + \left[ E\left( \sum_{i=1}^{n} X_i \right) \right]^2 = $$ $$ = E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 \right) - \left[ E \left( \sum_{i=1}^{n} X_i \right) \right] ^2 $$

which is the right hand side of the very first equivalence of the comment of Macro. In the end it's like adding a passage in it:

$$ {\rm var} \left( \sum_{i=1}^{n} X_i \right) = E \left( \left[ \sum_{i=1}^{n} X_i - E\left( \sum_{i=1}^{n} X_i \right)\right]^2 \right) = E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 \right) - \left[ E \left( \sum_{i=1}^{n} X_i \right) \right] ^2$$