Is it possible for two independent variables to be correlated, by chance?

Question

I'm reading Introduction to Probability, 2nd Edition by Tsitsiklis and Bertsekas.

On page 218, they write "Thus, if X and Y are independent, they are also uncorrelated."

Is it possible that there could be two variables where the value of one tends to increase as the value of the other increases, even if they come from totally unrelated and disconnected processes?

Answer 1

The question is confusing, and so maybe misinterpreted by some commenters and answerers. In your citation, there is two random variables $X$ and $Y$ which are independent. Then, there is a theorem saying that they are uncorrelated. It also have an easy proof, which you can find in many probability texts.

But this do not mean that if you have a sample $(X_1,Y_1), \dotsc, (X_n,Y_n)$ from $(X,Y)$, that the sample correlation coefficient will be zero! which is what the answer by @Nutle explains. But, if $n$ is large, the sampling distribution of that correlation coefficient will be concentrated close to zero.

So, yes, samples from two independent variables can seem to be correlated, by chance. Especially if $n$ is small. That just means that you risk having a type I error.

Answer 2

It's a matter of change.

Yes, it's always possible that on timepoint x, two independent variables can increase or decrease at the same time. At timepoint x it looks like these variables are dependent.

However, if the data is independent they follow their own trends in most of the timepoints.

Answer 3

Yes, and to add to all previous answers and comments, you can easily simulate the accidence.

I.e., this simple R example:

require(magrittr)
lapply(1:1000, function(b){
  set.seed(b)
  cor(rnorm(100), rnorm(100))
}) %>% do.call(c,.) %>% abs %>% max

Out of 1000 random draws there was a case when two independent normal random variables had a (weak) correlation of $\pm 0.34$.

This tends to decrease, of course, to zero, with increasing the size of the sample.

Answer 4

If X and Y are independent from each other, it can be proven that the correlation coefficient must be zero. However, there are two caveats:

1. As others point out, just because you have two populations that are uncorrelated, it does not necessarily mean that the samples drawn from the two populations will also be uncorrelated. Remember that there is a thing called "variance."

2. The reverse is not necessarily true. That is, two perfectly uncorrelated variables are not necessarily independent from each other. Correlation only measures the linear relationship. Just look at the chart below from Wikipedia: The last row shows how two perfectly uncorrelated variables can have non-linear relationship and thus are dependent.