Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Suppose we test the Pearson correlation between variable $x$ and $y$ in groups $A$ and $B$. Is it possible for the $(x,y)$ correlation to be significant in each of $A$ and $B$, but non-significant when data from both groups are combined? In this case, could you please provide an explanation for that.

share|improve this question
add comment

2 Answers

up vote 21 down vote accepted

Yes, it is possible and it could happen all sorts of ways. One obvious example is when membership of A and B is chosen in some way that reflects the values of x and y. Other examples are possible, eg @Macro's comment suggests an alternative possibility.

Consider the example below, written in R. x and y are iid standard normal variables, but if I allocate them to groups based on the relative values of x and y I get the siutation you name. Within group A and group B there is strong statistically significant correlation between x and y, but if you ignore the grouping structure there is no correlation.

enter image description here

> library(ggplot2)
> x <- rnorm(1000)
> y <- rnorm(1000)
> Group <- ifelse(x>y, "A", "B")
> cor.test(x,y)

        Pearson's product-moment correlation

data:  x and y 
t = -0.9832, df = 998, p-value = 0.3257
alternative hypothesis: true correlation is not equal to 0 
95 percent confidence interval:
 -0.09292  0.03094 
sample estimates:
     cor 
-0.03111 

> cor.test(x[Group=="A"], y[Group=="A"])

        Pearson's product-moment correlation

data:  x[Group == "A"] and y[Group == "A"] 
t = 11.93, df = 487, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0 
95 percent confidence interval:
 0.4040 0.5414 
sample estimates:
   cor 
0.4756 

> cor.test(x[Group=="B"], y[Group=="B"])

        Pearson's product-moment correlation

data:  x[Group == "B"] and y[Group == "B"] 
t = 9.974, df = 509, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0 
95 percent confidence interval:
 0.3292 0.4744 
sample estimates:
   cor 
0.4043 
> qplot(x,y, color=Group)
share|improve this answer
    
+1. This is a very clever example that hadn't occurred to me. –  Macro Jun 17 '12 at 2:45
add comment

One possibility is that the effects may be going in different directions in each group and are cancelled out when you aggregate them. This is also related to how, when you leave out an important interaction term in a regression model, the main effects can be misleading.

For example, suppose in group $\rm A$, the true relationship between the response $y_i$ and the predictor $x_i$ is:

$$ E(y_i|x_i, {\rm Group \ A}) = 1 + x_i $$

and in group $\rm B$,

$$ E(y_i|x_i, {\rm Group \ B}) = 1 - x_i $$

Suppose group membership is distributed so that $$P({\rm Group \ A}) = 1-P( {\rm Group \ B}) = p$$ Then, if you marginalize over the group membership and calculate $E(y_i|x_i)$ by Law of Total Expectation you get

\begin{align*} E(y_i | x_i) = E( E(y_i|x_i,{\rm Group}) ) &= p(1+ x_i) + (1-p)(1-x_i) \\ &= p + px_i + 1 - x_i - p + px_i \\ &= 1 - x_i(2p-1) \end{align*}

Therefore, if $p = 1/2$, $E(y_i | x_i) = 1$ and does not depend on $x_i$ at all. So, there is a relationship within both groups but, when you aggregate them, there is no relationship. In other words, for a randomly selected individual in the population, whose group membership we don't know, there will, on average, be no relationship between $x_i$ and $y_i$. But, within each group there is.

Any example where the value of $p$ perfectly balances the effect sizes within each group would also lead to this result - this was just this toy example to make the calculations easy :)

Note: With normal errors, significance of a linear regression coefficient is equivalent to significance of the Pearson's correlation, so this example highlights one explanation for what you're seeing.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.