3
$\begingroup$

$\newcommand{\Var}{\operatorname{Var}}\newcommand{Cov}{\operatorname{Cov}}$Mathematically,

$\Var(X + Y) = \Var(X) + \Var(Y) + 2\Cov(X,Y)$ and $\Var(X - Y) = \Var(X) + \Var(Y) - 2\Cov(X,Y)$

This implies that the variance of the sum of two positively correlated random variables is always larger than their difference. However, if the variables are uncorrelated, then the variance of their sum equals the variance of their difference.

I am having trouble understanding intuitively how

  1. The variances are equal if the variables are uncorrelated
  2. The variance of the sum is larger than the variance of the difference if they are correlated
$\endgroup$
4
  • $\begingroup$ "This implies that the variance of the sum of two correlated random variables is always larger than their difference" ... two variables that are negatively correlated are still correlated; in that situation the covariance will be negative and Var(X-Y) > Var(X+Y) $\endgroup$
    – Glen_b
    Dec 16, 2018 at 23:13
  • 3
    $\begingroup$ This does NOT imply the variance of the sum is always larger than the variance of their difference. That would be true only if the covariance is positive. $\endgroup$ Dec 17, 2018 at 0:58
  • $\begingroup$ ^ Right thank you both for clarifying. I overlooked that fact in my original post- I added an edit for clarity as I was referring to the case where two variables are positively correlated. $\endgroup$
    – Hank Lin
    Dec 17, 2018 at 14:01
  • $\begingroup$ Note that if you flip a variable around (attach a negative sign to it) you don't change its spread. $\endgroup$
    – Glen_b
    Dec 20, 2018 at 12:36

2 Answers 2

4
$\begingroup$

Let $Z = -Y$ so that $X-Y = X+Z$. Then, $$\operatorname{var}(Z)= (-1)^2\operatorname{var}(Y) = \operatorname{var}(Y)$$ and $$\operatorname{cov}(X,Z) = \operatorname{cov}(X,-Y) = -\operatorname{cov}(X,Y)$$ so that \begin{align} \operatorname{var}(X-Y) &= \operatorname{var}(X+Z)\\ &= \operatorname{var}(X)+\operatorname{var}(Z) + 2\operatorname{cov}(X,Z)\\ &= \operatorname{var}(X)+\operatorname{var}(Y) - 2\operatorname{cov}(X,Y) \end{align} In short, \begin{align} \operatorname{var}(X+Y) &= \operatorname{var}(X) + \operatorname{var}(Y) + 2\operatorname{cov}(X,Y)\\ \operatorname{var}(X-Y) &= \operatorname{var}(X) + \operatorname{var}(Y) - 2\operatorname{cov}(X,Y) \end{align} really are the same formula, and which of the two variances is larger depends entirely on whether $\operatorname{cov}(X,Y)$ is positive or negative. In the case when $\operatorname{cov}(X,Y)$ equals $0$ (i.e. $X$ and $Y$ are uncorrelated random variables), $\operatorname{var}(X+Y)$ equals $\operatorname{var}(X) + \operatorname{var}(Y)$ as you have already noted.

As to intuition regarding why $\operatorname{var}(X+Y) > \operatorname{var}(X-Y)$ when $\operatorname{cov}(X,Y) > 0$, a geometric viewpoint might help. $X$ and $Y$ can be regarded as vectors of lengths $\sigma_X = \sqrt{\operatorname{var}(X)}$ and $\sigma_X = \sqrt{\operatorname{var}(Y)}$ that are pointed roughly in the same direction when $\operatorname{cov}(X,Y) > 0$ and in roughly opposite direction when $\operatorname{cov}(X,Y) < 0$. Now, one would expect intuitively that the (vector) sum of two vectors pointing in roughly the same direction is a longer vector than the (vector) difference of the two vectors, no? A crudely drawn picture of the difference vector and the sum vector is in the diagram below.

Badly drawn diagram

Note that in the left-hand figure above, the vectors $X$ and $Y$ are shown as being at an acute angle $\theta$ and thus are "pointed in roughly the same direction". In fact, the correlation coefficient $\rho$ of (random variables) $X$ and $Y$ is $\cos(\theta) > 0$. Now, in a triangle with vertices $A, B, C$, and opposite sides of lengths $a, b$ and $c$ respectively, we have the following result familiar from elementary geometry/trigonometry $$c^2 = a^2 + b^2 - 2ab\cos(\angle C)$$ which is equivalent to \begin{align} \sigma_{X-Y}^2 &= \sigma_X^2 + \sigma_Y^2 - 2\sigma_X\sigma_Y\cos(\theta)\\ \operatorname{var}(X-Y) &= \operatorname{var}(X) + \operatorname{var}(Y) -2 \rho \sigma_X\sigma_Y\\ \operatorname{var}(X-Y) &= \operatorname{var}(X) + \operatorname{var}(Y) -2 \operatorname{cov}(X,Y) \end{align} On the other hand, as shown in the right-hand figure above, for vector sums, the included angle $\pi-\theta$ is obtuse, and so $\cos(\pi-\theta) = -\cos(\theta) < 0$. So, we get that \begin{align} \sigma_{X+Y}^2 &= \sigma_X^2 + \sigma_Y^2 + 2\sigma_X\sigma_Y\cos(\theta)\\ \operatorname{var}(X+Y) &= \operatorname{var}(X) + \operatorname{var}(Y) +2 \operatorname{cov}(X,Y), \end{align} that is, $\operatorname{var}(X+Y) > \operatorname{var}(X-Y)$ for positively correlated random variables.

$\endgroup$
1
  • $\begingroup$ Wow beautiful answer- thank you for your thoroughness. I learned a lot! $\endgroup$
    – Hank Lin
    Dec 17, 2018 at 14:31
1
$\begingroup$
  1. Consider $X$ and $Y$ as uniform random variables between $0$-$1$. Sum of them varies in $[0,2]$ while difference of them varies in $[-1,1]$. Both have the same range length. The length of their range is not equal to variance of course, but it is an indicator of variability in some sense.

  2. The variance of sum is not larger. You assume $cov(X,Y)>0$, which is not true in general.

$\endgroup$
2
  • $\begingroup$ Great thank you- your first point makes a lot of sense. And for the second point, right I was assuming that the covariance was positive. So I guess the sum is not strictly larger than the difference in all cases, but why is it larger in this particular case? Do you have a similar intuitive explanation? $\endgroup$
    – Hank Lin
    Dec 16, 2018 at 20:53
  • $\begingroup$ Hi: Take the case where the covariance is positive. Then differencing the random variables cancels out some of that correlation similarity ( this is really a re-wording of Dilip's point ) and makes the overall variance less than in the case where you sum the same two random variables. $\endgroup$
    – mlofton
    Dec 17, 2018 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.