2
$\begingroup$

$\newcommand{\Var}{\operatorname{Var}}\newcommand{Cov}{\operatorname{Cov}}$Mathematically,

$\Var(X + Y) = \Var(X) + \Var(Y) + 2\Cov(X,Y)$ and $\Var(X - Y) = \Var(X) + \Var(Y) - 2\Cov(X,Y)$

This implies that the variance of the sum of two positively correlated random variables is always larger than their difference. However, if the variables are uncorrelated, then the variance of their sum equals the variance of their difference.

I am having trouble understanding intuitively how

  1. The variances are equal if the variables are uncorrelated
  2. The variance of the sum is larger than the variance of the difference if they are correlated
$\endgroup$
  • $\begingroup$ "This implies that the variance of the sum of two correlated random variables is always larger than their difference" ... two variables that are negatively correlated are still correlated; in that situation the covariance will be negative and Var(X-Y) > Var(X+Y) $\endgroup$ – Glen_b Dec 16 '18 at 23:13
  • 3
    $\begingroup$ This does NOT imply the variance of the sum is always larger than the variance of their difference. That would be true only if the covariance is positive. $\endgroup$ – Michael Hardy Dec 17 '18 at 0:58
  • $\begingroup$ ^ Right thank you both for clarifying. I overlooked that fact in my original post- I added an edit for clarity as I was referring to the case where two variables are positively correlated. $\endgroup$ – hlinee Dec 17 '18 at 14:01
  • $\begingroup$ Note that if you flip a variable around (attach a negative sign to it) you don't change its spread. $\endgroup$ – Glen_b Dec 20 '18 at 12:36
3
$\begingroup$

Let $Z = -Y$ so that $X-Y = X+Z$. Then, $$\operatorname{var}(Z)= (-1)^2\operatorname{var}(Y) = \operatorname{var}(Y)$$ and $$\operatorname{cov}(X,Z) = \operatorname{cov}(X,-Y) = -\operatorname{cov}(X,Y)$$ so that \begin{align} \operatorname{var}(X-Y) &= \operatorname{var}(X+Z)\\ &= \operatorname{var}(X)+\operatorname{var}(Z) + 2\operatorname{cov}(X,Z)\\ &= \operatorname{var}(X)+\operatorname{var}(Y) - 2\operatorname{cov}(X,Y) \end{align} In short, \begin{align} \operatorname{var}(X+Y) &= \operatorname{var}(X) + \operatorname{var}(Y) + 2\operatorname{cov}(X,Y)\\ \operatorname{var}(X-Y) &= \operatorname{var}(X) + \operatorname{var}(Y) - 2\operatorname{cov}(X,Y) \end{align} really are the same formula, and which of the two variances is larger depends entirely on whether $\operatorname{cov}(X,Y)$ is positive or negative. In the case when $\operatorname{cov}(X,Y)$ equals $0$ (i.e. $X$ and $Y$ are uncorrelated random variables), $\operatorname{var}(X+Y)$ equals $\operatorname{var}(X) + \operatorname{var}(Y)$ as you have already noted.

As to intuition regarding why $\operatorname{var}(X+Y) > \operatorname{var}(X-Y)$ when $\operatorname{cov}(X,Y) > 0$, a geometric viewpoint might help. $X$ and $Y$ can be regarded as vectors of lengths $\sigma_X = \sqrt{\operatorname{var}(X)}$ and $\sigma_X = \sqrt{\operatorname{var}(Y)}$ that are pointed roughly in the same direction when $\operatorname{cov}(X,Y) > 0$ and in roughly opposite direction when $\operatorname{cov}(X,Y) < 0$. Now, one would expect intuitively that the (vector) sum of two vectors pointing in roughly the same direction is a longer vector than the (vector) difference of the two vectors, no? A crudely drawn picture of the difference vector and the sum vector is in the diagram below.

Badly drawn diagram

Note that in the left-hand figure above, the vectors $X$ and $Y$ are shown as being at an acute angle $\theta$ and thus are "pointed in roughly the same direction". In fact, the correlation coefficient $\rho$ of (random variables) $X$ and $Y$ is $\cos(\theta) > 0$. Now, in a triangle with vertices $A, B, C$, and opposite sides of lengths $a, b$ and $c$ respectively, we have the following result familiar from elementary geometry/trigonometry $$c^2 = a^2 + b^2 - 2ab\cos(\angle C)$$ which is equivalent to \begin{align} \sigma_{X-Y}^2 &= \sigma_X^2 + \sigma_Y^2 - 2\sigma_X\sigma_Y\cos(\theta)\\ \operatorname{var}(X-Y) &= \operatorname{var}(X) + \operatorname{var}(Y) -2 \rho \sigma_X\sigma_Y\\ \operatorname{var}(X-Y) &= \operatorname{var}(X) + \operatorname{var}(Y) -2 \operatorname{cov}(X,Y) \end{align} On the other hand, as shown in the right-hand figure above, for vector sums, the included angle $\pi-\theta$ is obtuse, and so $\cos(\pi-\theta) = -\cos(\theta) < 0$. So, we get that \begin{align} \sigma_{X+Y}^2 &= \sigma_X^2 + \sigma_Y^2 + 2\sigma_X\sigma_Y\cos(\theta)\\ \operatorname{var}(X+Y) &= \operatorname{var}(X) + \operatorname{var}(Y) +2 \operatorname{cov}(X,Y), \end{align} that is, $\operatorname{var}(X+Y) > \operatorname{var}(X-Y)$ for positively correlated random variables.

$\endgroup$
  • $\begingroup$ Wow beautiful answer- thank you for your thoroughness. I learned a lot! $\endgroup$ – hlinee Dec 17 '18 at 14:31
1
$\begingroup$
  1. Consider $X$ and $Y$ as uniform random variables between $0$-$1$. Sum of them varies in $[0,2]$ while difference of them varies in $[-1,1]$. Both have the same range length. The length of their range is not equal to variance of course, but it is an indicator of variability in some sense.

  2. The variance of sum is not larger. You assume $cov(X,Y)>0$, which is not true in general.

$\endgroup$
  • $\begingroup$ Great thank you- your first point makes a lot of sense. And for the second point, right I was assuming that the covariance was positive. So I guess the sum is not strictly larger than the difference in all cases, but why is it larger in this particular case? Do you have a similar intuitive explanation? $\endgroup$ – hlinee Dec 16 '18 at 20:53
  • $\begingroup$ Hi: Take the case where the covariance is positive. Then differencing the random variables cancels out some of that correlation similarity ( this is really a re-wording of Dilip's point ) and makes the overall variance less than in the case where you sum the same two random variables. $\endgroup$ – mlofton Dec 17 '18 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.