Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Is it (always) true that $$\mathrm{Var}\left(\sum\limits_{i=1}^m{X_i}\right) = \sum\limits_{i=1}^m{\mathrm{Var}(X_i)} \>?$$

share|improve this question

3 Answers 3

up vote 19 down vote accepted

The answer to your question is "Sometimes, but not in general".

To see this let $X_1, ..., X_n$ be random variables (with finite variances). Then,

$$ {\rm var} \left( \sum_{i=1}^{n} X_i \right) = E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 \right) - \left[ E\left( \sum_{i=1}^{n} X_i \right) \right]^2$$

Now note that $(\sum_{i=1}^{n} a_i)^2 = \sum_{i=1}^{n} \sum_{j=1}^{n} a_i a_j $, which is clear if you think about what you're doing when you calculate $(a_1+...+a_n) \cdot (a_1+...+a_n)$ by hand. Therefore,

$$ E \left( \left[ \sum_{i=1}^{n} X_i \right]^2 \right) = E \left( \sum_{i=1}^{n} \sum_{j=1}^{n} X_i X_j \right) = \sum_{i=1}^{n} \sum_{j=1}^{n} E(X_i X_j) $$

similarly,

$$ \left[ E\left( \sum_{i=1}^{n} X_i \right) \right]^2 = \left[ \sum_{i=1}^{n} E(X_i) \right]^2 = \sum_{i=1}^{n} \sum_{j=1}^{n} E(X_i) E(X_j)$$

so

$$ {\rm var} \left( \sum_{i=1}^{n} X_i \right) = \sum_{i=1}^{n} \sum_{j=1}^{n} \big( E(X_i X_j)-E(X_i) E(X_j) \big) = \sum_{i=1}^{n} \sum_{j=1}^{n} {\rm cov}(X_i, X_j)$$

by the definition of covariance.

Now regarding Does the variance of a sum equal the sum of the variances?:

  • If the variables are uncorrelated, yes: that is, ${\rm cov}(X_i,X_j)=0$ for $i\neq j$, then $$ {\rm var} \left( \sum_{i=1}^{n} X_i \right) = \sum_{i=1}^{n} \sum_{j=1}^{n} {\rm cov}(X_i, X_j) = \sum_{i=1}^{n} {\rm cov}(X_i, X_i) = \sum_{i=1}^{n} {\rm var}(X_i) $$

  • If the variables are correlated, no, not in general: For example, suppose $X_1, X_2$ are two random variables each with variance $\sigma^2$ and ${\rm cov}(X_1,X_2)=\rho$ where $0 < \rho <\sigma^2$. Then ${\rm var}(X_1 + X_2) = 2(\sigma^2 + \rho) \neq 2\sigma^2$, so the identity fails.

  • but it is possible for certain examples: Suppose $X_1, X_2, X_3$ have covariance matrix $$ \left( \begin{array}{ccc} 1 & 0.4 &-0.6 \\ 0.4 & 1 & 0.2 \\ -0.6 & 0.2 & 1 \\ \end{array} \right) $$ then ${\rm var}(X_1+X_2+X_3) = 3 = {\rm var}(X_1) + {\rm var}(X_2) + {\rm var}(X_3)$

Therefore if the variables are uncorrelated then the variance of the sum is the sum of the variances, but converse is not true in general.

share|improve this answer
3  
This is very cool. –  Kris Harper Jun 27 '12 at 12:34
1  
Thank you for providing such a detailed and clear explanation. –  Abe Jun 27 '12 at 17:44
    
Regarding the example covariance matrix, is the following correct: the symmetry between the upper right and lower left triangles reflects the fact that $\text{cov}(X_i,X_j)=\text{cov}(X_j,X_i)$, but the symmetry between the upper left and the lower right (in this case that $\text{cov}(X_1, X_2) = \text{cov}(X_2,X_3) = 0.3$ is just part of the example, but could be replaced with two different numbers that sum to $0.6$ e.g., $\text{cov}(X_1, X_2) = a$ and $\text{cov}(X_2,X,3) = 0.6 -a$? Thanks again. –  Abe Jun 27 '12 at 17:56
1  
thanks. I proposed an edit to make it more 'textbook' like / dummy resistant. –  Abe Jun 27 '12 at 18:06

$$\text{Var}\bigg(\sum_{i=1}^m X_i\bigg) = \sum_{i=1}^m \text{Var}(X_i) + 2\sum_{i\lt j} \text{Cov}(X_i,X_j).$$

So, if the covariances average to $0$, which would be a consequence if the variables are pairwise uncorrelated or if they are independent, then the variance of the sum is the sum of the variances.

An example where this is not true: Let $\text{Var}(X_1)=1$. Let $X_2 = X_1$. Then $\text{Var}(X_1 + X_2) = \text{Var}(2X_1)=4$.

share|improve this answer
    
(+1) For completeness. –  cardinal Jun 26 '12 at 23:27
    
It will rarely be true for sample variances. –  DWin Jun 27 '12 at 2:35
    
@DWin, "rare" is an understatement - if the $X$s have a continuous distribution, the probability that the sample variance of the sum is equal to the sum of the sample variances in exactly 0 :) –  Macro Jun 27 '12 at 13:41

Yes, if each pair of the $X_i$'s are uncorrelated, this is true.

See the explanation on Wikipedia

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.