# Why are the correlations in two groups less than the correlation when the groups are combined?

I have been running correlations for a set of data and several subsamples. During this analysis I ran into a situation where the $r^2$ for two groups was smaller in each individual group as opposed to when they are grouped together.

• Is there any straight forward explanation for how this could happen?
-

Here are just a couple of ideas:

• Range restriction is one explanation. Check out this simulation; and this explanation
• Correlated group mean differences is another related idea. Say group 1 has a mean two standard deviations higher than group 2 on both X and Y, but that there is no correlation between X and Y within each group. When you combine the two groups there would be a strong correlation.

And just for fun, here's a little R simulation

# Setup Data
x1 <- rnorm(200, 0, 1)
x2 <- rnorm(200, 2, 1)
y1 <- rnorm(200, 0, 1)
y2 <- rnorm(200, 2, 1)
grp <- rep(1:2, each=200)
x <- data.frame(grp, x=c(x1,x2), y=c(y1,y2))

# Plot
library(lattice)
xyplot(y~x, group=grp, data=x)

# Correlations
cor(x1, y1)
cor(x2, y2)
cor(x$x, x$y)


Which produced these three correlations respectively on my run of the simulation

[1] 0.1248730
[1] 0.1027219
[1] 0.56244


And the following graph

-
+1 The graph says it all. When you separate the pink and blue dots enough, you can bring the correlation as close to $\pm 1$ as you like. –  whuber Jun 3 '11 at 16:57
Thanks so much for this. The effect in my data is not quite that strong, but this seems reasonable. –  MudPhud Jun 3 '11 at 19:19