6
$\begingroup$

If I take a set of measurements and test correlation of variable $A$ vs variable $B$ and get a significant correlation, that makes sense to me. But what if further analysis reveals that of those factors, there is only a significant positive correlation within one group, and that group is over-represented. Is the global correlation still valid, or is it, upon more detailed inspection a sample-bias effect?

Here is some graphs to explain:

The global correlation

global correlation

The group separated correlations

grouping separated correlation

$\endgroup$
  • $\begingroup$ Just a little comment: it looks like the overall correlation might partly be explained by the fact that units belonging to group A all have lower scores whereas considering valueB alone doesn't help to separate the three groups. $\endgroup$ – chl May 26 '11 at 8:34
  • $\begingroup$ I think if you give us more content (i.e., what do the groups mean) you will get better advice on what to do. $\endgroup$ – Henrik May 26 '11 at 21:09
7
$\begingroup$

I agree with JMS advice, that the answer is totally context dependent.

But what you are looking at may also be considered a moderation effect.

In statistics, moderation occurs when the relationship between two variables depends on a third variable.

(quoted from wikipedia)

A moderation is statistically significant if in a multiple regression analyses the interaction of the predictor with the third variable is significant.

$\endgroup$
8
$\begingroup$

Are you familiar with Simpson's paradox? This would seem to be what you're observing here.

Edit: I didn't answer your question :) What exactly you should do is to some degree context dependent (Are the groups meaningful? Does this represent a problem in the study design? etc). At the very least you should report both results IMO.

$\endgroup$
5
$\begingroup$

The previous comments are all good, but with group sample sizes of 5, 7, and 11, I wouldn't trust any of their correlations as far as I could throw them. You'll need to give the overall r a wide confidence interval as well. btw Nice job on the graph.

$\endgroup$
  • 1
    $\begingroup$ +1, good point. I took it as an example, not the actual data, but reading again the OP certainly didn't say that. $\endgroup$ – JMS May 27 '11 at 3:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.