Puzzling negative correlation

Question

Question edited with the correct code - apologies

I have observed a somewhat puzzling negative correlation.

In code (R) (mathematical formulation below)

set.seed(123)
N=1000
R1=vector('double',N)
R2=vector('double',N)
x=rnorm(100)
for (k in 1:N){
  y1=x+rnorm(100)
  y2=x+rnorm(100)
  X=c(x,x)
  Y=c(y1,y2)
  R1[k]=cor(y1,y2)
  R2[k]=var(X)/var(Y)  
}
cor.test(R1,R2)

    Pearson's product-moment correlation

data:  R1 and R2
t = -7.2284, df = 998, p-value = 9.704e-13
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 -0.2811533 -0.1633130
sample estimates:
       cor 
-0.2230479 

It turns out that R1 and R2 are very close to each other but negatively correlated. This is only the case when x is kept out of the loop (i.e. fix across experiments).

Would you know why?

Mathematical formulation

Let

\begin{align} & x_i \sim \operatorname N(0,1),\quad i=1,\ldots,100 \quad (\sim\text{indicates i.i.d.}) \\[6pt] & y_{1,i} = x_i + u_i, \quad u_i\sim\operatorname N(0,1) \\[6pt] & y_{2,i} = x_i + v_i, \quad v_i\sim\operatorname N(0,1) \end{align}

We can arrange these random variables as vectors:

\begin{align} & \mathbf x = [x_i]_{i=1,\ldots,100} \\[6pt] & \mathbf y_1 = [y_{1,i}]_{i=1,\ldots,100} \\[6pt] & \mathbf y_2 = [y_{2,i}]_{i=1,\ldots,100} \end{align}

And we now create the concatenated vectors as follows:

\begin{align} & \mathbf X = [\mathbf x, \mathbf x] \\[6pt] & \mathbf Y = [\mathbf y_1, \mathbf y_2] \end{align}

It turns out that the Pearson's correlation between $\mathbf y_1$ and $\mathbf y_2$ is very close to $\operatorname{Var}(X)/\operatorname{Var}(Y).$ However, the two are negatively correlated when we repeat the process of realising $y_{1,i}$ and $y_{2,i}$ while maintaining $x_i$ constant across realisations.

Would you know why?

Answer 1

R2 is supposed to be defined as Var(y)/Var(x) to get a positive correlation. The sample correlation is given by cov(x,y)*sd(y)/sd(x).

Answer 2

• $Var(x)$ and $Var(X)$ and $Cov(y1,y2)$ usually about $1$, with the first two constant through your simulation and almost equal to each other (another run or with a different seed would give a different constant value).
• $Var(y_1)$, $Var(y_2)$, $\sqrt{Var(y1)Var(y2)}$ and $Var(Y)$ are usually usually be about $1.25$ with the last two almost equal.
• So $R_1=Var(X)/Var(Y)$ should usually be about $0.8$.
• and $R_2=Cor(y_1,y_2)=Cov(y_1,y_2)/\sqrt{Var(y1)Var(y2)}$ also usually about 0.8.

Empirically you can try

plot(R1,R2)

to see that they are both not far from $0.8$ and that there is not much relationship between them. The small negative correlation comes from:

• $\sqrt{Var(y1)Var(y2)}$ and $Cov(y_1,y_2)$ are strongly positively correlated
• $\sqrt{Var(y1)Var(y2)}$ and $Cov(y_1,y_2)/\sqrt{Var(y1)Var(y2)}$ are weakly positively correlated (the division in the second term removes most but not all of the correlation)
• $1/\sqrt{Var(y1)Var(y2)}$ and $Cov(y_1,y_2)/\sqrt{Var(y1)Var(y2)}$ are weakly negatively correlated (the reciprocal of the first term tends to reverse the correlation)
• $1/Var(Y)$ and $Cov(y_1,y_2)/\sqrt{Var(y1)Var(y2)}$ are weakly negatively correlated (replacing the first term with one almost equal to it)
• $R_1=Var(X)/Var(Y)$ and $R_2=Cov(y_1,y_2)/\sqrt{Var(y1)Var(y2)}$ are weakly negatively correlated (multiplying the first term by something which is a positive constant through the simulation does not change the correlation)