If correlation between two variables is affected by a factor, how should I evaluate this correlation?

Question

Here is my example. Supose we evaluate a characteristic using two different methods (a and b) and we want to study if both methods performs in a same way. We also know that these two measures have been recorded from two different groups, and the mean values for each one of these groups are highly different. Our data set could be as follows:

a <- c(22,34,56,62,27,53)
b <- c(42.5,43,58.6,55,31.2,51.75)
group <- factor(c(1,1,2,2,1,2), labels=c('bad','good'))
dat <- data.frame(a, b, group)

The association between a and b could be calculated as:

lm1 <- lm(a ~ b, data=dat)
summary(lm1)

Call:
lm(formula = a ~ b, data = dat)

Residuals:
      1       2       3       4       5       6 
-13.810  -2.533  -3.106   8.103   7.541   3.806 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)  
(Intercept) -25.6865    19.7210  -1.302   0.2627  
b             1.4470     0.4117   3.514   0.0246 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Residual standard error: 9.271 on 4 degrees of freedom
Multiple R-squared: 0.7554, Adjusted R-squared: 0.6942 
F-statistic: 12.35 on 1 and 4 DF,  p-value: 0.02457

As we can see, it seems to be a high association between both measures. However, if we perform the same analysis for each group separately, this association disappears.

lm2 <- lm(a ~ b, data=dat, subset=dat$class=='bad')
summary(lm2)

Call:
lm(formula = a ~ b, data = dat, subset = dat$group == "bad")

Residuals:
      1       2       5 
-6.0992  5.8407  0.2584 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept)  22.9931    35.1657   0.654    0.631
b             0.1201     0.8953   0.134    0.915

Residual standard error: 8.449 on 1 degrees of freedom
Multiple R-squared: 0.01769,    Adjusted R-squared: -0.9646 
F-statistic: 0.01801 on 1 and 1 DF,  p-value: 0.915

and,

lm3 <- lm(a ~ b, data=dat, subset=dat$class=='good')
summary(lm3)

Call:
lm(formula = a ~ b, data = dat, subset = dat$group == "good")

Residuals:
     3      4      6 
-2.394  5.047 -2.652 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept)  34.9361    70.4238   0.496    0.707
FIV           0.4003     1.2761   0.314    0.806

Residual standard error: 6.184 on 1 degrees of freedom
Multiple R-squared: 0.08959,    Adjusted R-squared: -0.8208 
F-statistic: 0.09841 on 1 and 1 DF,  p-value: 0.8065 

How should we assess the association between the two methods? We should take into account the group factor? Maybe it is a trivial question, but I have doubts about how to deal with this problem.

Answer 1

This might be a case of locally uncorrelated, but globally correlated variables. The variance in each group might be limited because of group homogeneity, therefore there is no evidence for a relationship within each group. But globally, with the full variance, the relationship can be strong. A schematic illustration of the joint distribution within three groups, and the resulting global joint distribution:

Edit: Your question also seems to be if the global correlation is still "real", even if the theoretical correlation within each group is 0. Random variables are defined on a probability space $<\Omega, P>$ where $\Omega$ is the set of all outcomes (think of different observable persons in your case), and $P$ is a probability measure. If your natural population $\Omega$ includes members from all groups, then: yes, the variables are "really" correlated. Otherwise, if the members of different groups do not form a natural common $\Omega$, but each belong to separate populations, then: no, the variables are uncorrelated.

Answer 2

Therefore, it is important to evaluate whether the homogeneity of groups is due to low number of data, or actually these groups are quite homogeneous and different. In the first case, we could ensure the presence of a high correlation even when this has not been observed for each group separately.

But what would happen in the second case? If even using a large number of data we could not observe a correlation within each group, we could say that this correlation exist?

Perhaps the value of one of these measures will be only useful for predicting membership in one of the two groups, but not to predict the value of the other measure.