2
$\begingroup$

On pg. 44 of Introduction to Statistical Learning:

Here we create two correlated sets of numbers, x and y, and use the cor() function to compute the correlation between them.

x = rnorm(50)
y = x + rnorm(50, mean=50, sd=.1)
cor(x,y)
[1] 0.995

Observation: Playing around with this and related examples, I see the following:

Let $x$ be vector of length $n$ randomly sampled from a normal population with mean $0$ and standard deviation $1$. Let $y = x + z$ such that $z$ is a vector of length $n$ with sample mean $\bar{z}$ and sample standard deviation $s_z$.

  1. Then $x$ and $y$ tend to be correlated so long as $s_z$ is reasonably low (the lower it is, the more $x$ and $y$ tend to be correlated).

  2. I have also noticed that the sample mean of $z$ doesn't tend to affect the correlation between $x$ and $y$.

Question: Formally, why do (1) and (2) hold? (Really, I'm assuming that an understanding of (1) will automatically clarify (2)).

$\endgroup$
1
  • 1
    $\begingroup$ See our threads on computing covariance of sums and apply the definition of correlation. That gives a rigorous algebraic approach. Alternatively, drawing a picture of how $y$ is constructed from $x$ and $z$ will make the result immediately obvious. $\endgroup$
    – whuber
    Commented May 27, 2016 at 20:51

1 Answer 1

2
$\begingroup$

For illustration purpose, consider smaller vectors:

set.seed(101)
x = rnorm(5) # x = -0.3260365  0.5524619 -0.6749438  0.2143595  0.3107692

First, let's say $z$ is a vector of constant values, i.e., $s_z = 0$

z <- rep(50, 5)
y <- x + z

Now plot(x,y) will produce a perfect straight line and cor(x,y) will return 1 meaning perfect correlation between $x$ and $y$. It is because given an $x$ value, we can accurately predict $y$ value ($y = x + 50$).
Again, take random values of $z$ with mean $\bar z = 50$ and standard deviation $s_z = 5$. What this means is that now $y$ depends not only on the randomness of $x$, but also randomness of $z$. Larger SD of $z$ indicates the values are more distant from the mean, $50$. So if we try to predict $y$ from $y = x + 50$, as we did before, our prediction will be a lot less accurate meaning weaker correlation. Speaking of your second question, as it should have been clear by now, SD instead of mean affects the correlation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.