# Correlation significant in each group but non-significant over all?

Suppose we test the Pearson correlation between variable $x$ and $y$ in groups $A$ and $B$. Is it possible for the $(x,y)$ correlation to be significant in each of $A$ and $B$, but non-significant when data from both groups are combined? In this case, could you please provide an explanation for that.

Yes, it is possible and it could happen all sorts of ways. One obvious example is when membership of A and B is chosen in some way that reflects the values of x and y. Other examples are possible, eg @Macro's comment suggests an alternative possibility.

Consider the example below, written in R. x and y are iid standard normal variables, but if I allocate them to groups based on the relative values of x and y I get the siutation you name. Within group A and group B there is strong statistically significant correlation between x and y, but if you ignore the grouping structure there is no correlation.

> library(ggplot2)
> x <- rnorm(1000)
> y <- rnorm(1000)
> Group <- ifelse(x>y, "A", "B")
> cor.test(x,y)

Pearson's product-moment correlation

data:  x and y
t = -0.9832, df = 998, p-value = 0.3257
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
-0.09292  0.03094
sample estimates:
cor
-0.03111

> cor.test(x[Group=="A"], y[Group=="A"])

Pearson's product-moment correlation

data:  x[Group == "A"] and y[Group == "A"]
t = 11.93, df = 487, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.4040 0.5414
sample estimates:
cor
0.4756

> cor.test(x[Group=="B"], y[Group=="B"])

Pearson's product-moment correlation

data:  x[Group == "B"] and y[Group == "B"]
t = 9.974, df = 509, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.3292 0.4744
sample estimates:
cor
0.4043
> qplot(x,y, color=Group)

• +1. This is a very clever example that hadn't occurred to me. – Macro Jun 17 '12 at 2:45

One possibility is that the effects may be going in different directions in each group and are cancelled out when you aggregate them. This is also related to how, when you leave out an important interaction term in a regression model, the main effects can be misleading.

For example, suppose in group $\rm A$, the true relationship between the response $y_i$ and the predictor $x_i$ is:

$$E(y_i|x_i, {\rm Group \ A}) = 1 + x_i$$

and in group $\rm B$,

$$E(y_i|x_i, {\rm Group \ B}) = 1 - x_i$$

Suppose group membership is distributed so that $$P({\rm Group \ A}) = 1-P( {\rm Group \ B}) = p$$ Then, if you marginalize over the group membership and calculate $E(y_i|x_i)$ by Law of Total Expectation you get

\begin{align*} E(y_i | x_i) = E( E(y_i|x_i,{\rm Group}) ) &= p(1+ x_i) + (1-p)(1-x_i) \\ &= p + px_i + 1 - x_i - p + px_i \\ &= 1 - x_i(2p-1) \end{align*}

Therefore, if $p = 1/2$, $E(y_i | x_i) = 1$ and does not depend on $x_i$ at all. So, there is a relationship within both groups but, when you aggregate them, there is no relationship. In other words, for a randomly selected individual in the population, whose group membership we don't know, there will, on average, be no relationship between $x_i$ and $y_i$. But, within each group there is.

Any example where the value of $p$ perfectly balances the effect sizes within each group would also lead to this result - this was just this toy example to make the calculations easy :)

Note: With normal errors, significance of a linear regression coefficient is equivalent to significance of the Pearson's correlation, so this example highlights one explanation for what you're seeing.