4
$\begingroup$

Suppose I have a dataset consisting of $3N$ elements ($X,Y$ pairs) and I split it evenly into three subsets. I know the Pearson correlations between $(X,Y)$ in each of these subsets, and suppose they are all equal to some value $k$, i.e. $r_1 = r_2 = r_3 = k > 0$. Is it possible for the correlation $r$ between $(X,Y)$ in the entire dataset to be, say $2k$ (or any number $> k$).

I thought about this question for a while, I can easily see cases where the correlation $r$ will decrease or stay the same, however, I could not find a justification or a counter-example for it to become larger.

On the other hand I was trying to approach this problem through the linear regression point of view, and I can construct an example where three lines corresponding to each of the subsets have smaller slope than the line fitted to the entire set, I understand that slope and correlation are connected, however, I could not get the desired answer for the correlation of a full dataset from this.

$\endgroup$

1 Answer 1

5
$\begingroup$

Regardless of what $k$ might be, $r$ could have any value in the interval $(-1,1)$. Statistically, this asserts that the correlations of disjoint subgroups of a population do not provide any information about the correlation of the population.

This is easy to see geometrically. Take three clusters of points $(x_i^{(j)}, y_i^{(j)})$, $j=-1,0,1$, having any correlations whatsoever (and any counts, for that matter). Visualize translate them by the vector $j(\lambda,\lambda)$ (first case) or $j(\lambda,-\lambda)$ (second case). Translation does not change the correlations within the clusters. As $\lambda$ grows in size, this moves the first and third clusters away from the third along a line of slope $1$ in the first case or slope $-1$ in the second case. In the limit the clusters grow small compared to the separation $|\lambda|$, so the correlation must approach $1$ (in the first case) or $-1$ (in the second case).

Any intermediate correlation can be achieved with a modification of this approach: translate cluster $-1$ to any location and translate cluster $1$ to any other location. Moving them around continuously changes the overall correlation $r$ continuously. The previous construction shows $r$ can be arbitrarily close to $-1$ or $+1$. The Intermediate Value Theorem implies all intermediate values of $r$ will be attained as you continuously move the configuration from the first case to the second case.

Here, to illustrate, is a picture where each cluster consists of three uncorrelated points ($k=0$). They have been positioned along the line $y=-x$ so they are separated by about five times the cluster diameters, making the overall correlation nearly $-1$:

Figure 1

The point clusters are distinguished by color. The title displays $k$ and $r$, in that order.

$\endgroup$
2
  • $\begingroup$ Thank you for the insightful answer. I see now that what I was missing is the fact that the correlation within the cluster is invariant under translations. Can you recommend any reference or book that is helpful in building intuition behind questions like this? $\endgroup$
    – Pukki
    Commented Jun 27, 2017 at 15:59
  • $\begingroup$ Consider reading through our highest-voted threads on regression or correlation. Refining the searches with "intuition" might also be useful. $\endgroup$
    – whuber
    Commented Jun 27, 2017 at 16:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.