Taken from Practical Statistics for Medical Research where Douglas Altman writes in page 285:

...for any two quantities X and Y, X will be correlated with X-Y. Indeed, even if X and Y are samples of random numbers we would expect the correlation of X and X-Y to be 0.7

I tried this in R and it seems to be the case:

x <- rnorm(1000000, 10, 2)
y <- rnorm(1000000, 10, 2)
cor(x, x-y)

xu <- sample(1:100, size = 1000000, replace = T)
yu <- sample(1:100, size = 1000000, replace = T)
cor(xu, xu-yu)

Why is that? What is the theory behind this?

  • $\begingroup$ What part do you want an explanation for? Do you just want the simplified equation for the correlation that results because of the known correlation between x, and y, and covariance between x and x-y? Or, do you just want to know why there's any covariance here at all? $\endgroup$
    – John
    Commented Mar 6, 2013 at 11:52
  • $\begingroup$ Is this true for any $X$ and $Y$? Suppose $X$ and $Z$ are uncorrelated and let $Y=X-Z$. Then I suspect $X$ will not be correlated with $X-Y$. $\endgroup$
    – Henry
    Commented Aug 23, 2013 at 17:08

4 Answers 4


If $X$ and $Y$ are uncorrelated random variables with equal variance $\sigma^2$, then we have that $$\begin{align} \operatorname{var}(X-Y) &= \operatorname{var}(X) + \operatorname{var}(-Y)\\ &= \operatorname{var}(X) + \operatorname{var}(Y)\\ &=2\sigma^2,\\ \operatorname{cov}(X, X-Y) &= \operatorname{cov}(X,X) - \operatorname{cov}(X,Y) & \text{bilinearity of covariance operator}\\ &= \operatorname{var}(X) - 0 & 0 ~\text{because}~X ~\text{and}~ Y ~\text{are uncorrelated}\\ &= \sigma^2. \end{align}$$ Consequently, $$\rho_{X,X-Y} = \frac{\operatorname{cov}(X, X-Y)}{\sqrt{\operatorname{var}(X)\operatorname{var}(X-Y)}}= \frac{\sigma^2}{\sqrt{\sigma^2\cdot2\sigma^2}} = \frac{1}{\sqrt{2}}.$$ So, when you find $$\frac{\sum_{i=1}^n\left(x_i - \bar{x}\right) \left((x_i-y_i) - (\bar{x}-\bar{y})\right)}{ \sqrt{\sum_{i=1}^n\left(x_i - \bar{x}\right)^2 \sum_{i=1}^n\left((x_i-y_i) - (\bar{x}-\bar{y})\right)^2}} $$ the sample correlation of $x$ and $x-y$ for a large data set $\{(x_i,y_i)\colon 1 \leq i \leq n\}$ drawn from a population with these properties, which includes "random numbers" as a special case, the result tends to be close to the population correlation value $\frac{1}{\sqrt{2}} \approx 0.7071\ldots$

  • $\begingroup$ Could you please explain a bit more how cov(X,X)-cov(X,Y)=s^2 $\endgroup$
    – nostock
    Commented Mar 6, 2013 at 18:56
  • 6
    $\begingroup$ cov(X,X) is another name for var(X). cov(X,Y) = 0 since X and Y are assumed to be uncorrelated (hence covariance = 0). $\endgroup$ Commented Mar 6, 2013 at 19:15

A geometrical-statistical explanation.

Imagine you make an "inside-out" scatterplot where the $n$ subjects are the axes and the $2$ variables $X$ and $Y$ are the points. This is called a subject space plot (as opposed to usual variable space plot). Because there is only 2 points to plot, all dimensions in such a space except just any two arbitrary dimensions that are able to support the 2 points plus the origin, are redundant and can be safely dropped. And so we are left with a plane. We draw vector arrows from the origin to the points: these are our variables $X$ and $Y$ as vectors in the subject space of the data.

Now, if the variables were centered then, in a subject space, the cosine of the angle between their vectors is their correlation coefficient. On the pic below $X$ and $Y$ vectors are orthogonal: their $r=0$. Uncorrelatedness was a prerequisite outlined by @Dilip in their answer.

Also for variables centered, their vector lengths in a subject space are their standard deviations. On the pic, $X$ and $Y$ are of equal length, - equal variances was also a prerequisite made by @Dilip.

To draw the variable $X-Y$ or variable $X+Y$ we just use vector addition or subtraction that we've forgotten since school (move Y vector over to the end of X vector and invert direction in case of subtraction, - this is shown by grey arrows on the pic, - then draw a vector to where the grey arrow points).

It becomes very clear that the length of $X-Y$ or $X+Y$ vectors (the standard deviation of these variables) is, by Pythagorean theorem, $\sqrt{2\sigma^2}$, and the angle between $X$ and $X-Y$ or $X+Y$ is 45 degrees, which cosine - the correlation - is $0.707...$

enter image description here

  • 4
    $\begingroup$ A big +1 for sharing this approach. $\endgroup$
    – whuber
    Commented Mar 6, 2013 at 16:43
  • $\begingroup$ (+1) That is a very neat way of presenting this! $\endgroup$ Commented Mar 6, 2013 at 17:11
  • $\begingroup$ Ahh...pictures! (+1) Well done. :-) $\endgroup$
    – cardinal
    Commented Mar 11, 2013 at 13:45

I believe that there's a simple intuition based on symmetry here, too. Since X and Y have the same distributions and have a covariance of 0, the relationship of X ± Y with X should "explain" half of the variation in X ± Y; the other half should be explained by Y. So R2 should be 1/2, which means R is 1/√2 ≈ 0.707.

  • $\begingroup$ This seems like a nice intuition, but note that if $r^2=\frac 1 2$, the standard way to write $r$ would be $\sqrt{1/2}$, not $1/\sqrt 2$ which might confuse some people even if they are algebraically equivalent. $\endgroup$ Commented Mar 30, 2014 at 8:05
  • $\begingroup$ No, that really is not more standard. (If you need evidence, look up at the top answer. The 38 people who have already voted for it didn't quibble with the same notation.) $\endgroup$
    – denn333
    Commented Mar 30, 2014 at 16:19
  • $\begingroup$ I'm one of those 38 ;-). The question is, what will somebody whose algebra is fairly weak be most easily able to follow? If $r^2=1/2$, then it is easier to see that $r=\sqrt{1/2}$. $\endgroup$ Commented Mar 30, 2014 at 16:24

Here's a simple way to think about why there's a correlation here at all.

Imagine what goes on when you subtract two distributions. If the value of x is low then, on average, x - y will be a lower value than if the value of x is high. As x increases then x - y increase, on average, and thus, a positive correlation.

  • 4
    $\begingroup$ I don't think your statement is always true "There will always be a correlation between two random distributions when there's a mathematical relationship." e.g. x <- rnorm(1e6, 0,1) y <- rnorm(1e6, 0,1) $cor((x-y)^2,x-y) $ $\endgroup$ Commented Mar 6, 2013 at 13:34
  • 4
    $\begingroup$ @curious_cat: Or, perhaps to be even more evocative, drop the y altogether. :-) $\endgroup$
    – cardinal
    Commented Mar 11, 2013 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.