I have some data from two groups of subjects. Within each group, the correlation is calculated on some biological measures between any pair of two subjects. In other words, a group of $N$ subjects gives me $N(N-1)/2$ correlation coefficients, which show the similarity between any pair of two subjects. The biologist would like to compare the similarity between the two groups, and this is the starting point for him. So please don't ask why I can't directly work on the original measures. That is, with $N_1$ and $N_2$ subjects in the two groups respectively, we need to compare $N_1(N_1-1)/2$ correlation coefficients from the first group with $N_2(N_2-1)/2$ correlation coefficients in the second group.

First it would make sense to convert the correlation coefficients to $Z$-score based on Fisher transformation. A simple two-sample $t$-test is apparently problematic simply because those values in each group are not fully independent ($N-1$ values come from each subject). Does anyone have a good approach to handling this?

I came up with the following idea, but I'm not so sure about its validity. A group of $N$ subjects gives us $N(N-1)/2$ correlation coefficients, and each subject has $N-1$ correlation coefficients associated with him/her. If we create an R data frame in long format with all subjects' correlation coefficients, the number of rows would be $N(N-1)$, and each value is duplicated. This can be seen from the following simple example of 3 subjects in each group.

# for demo only: not real data
nSubj <- 3     # number of subjects in each group
val <- rnorm(choose(2*nSubj,2), 0, 1)  # Fisher Z-score
myDat <- data.frame(
      Subj=rep(paste('s', 1:(2*nSubj), sep=''), each = nSubj-1),  # subject column
      group=c(rep('g1', nSubj*(nSubj-1)), rep('g2', nSubj*(nSubj-1))),
      corr=rep(paste('cor', 1:(nSubj-1), sep=''), 2*nSubj),   # factor column
      y=c(val[c(1,2)], val[c(2,3)], val[c(1,3)], val[c(4,5)], val[c(5,6)], val[c(4,6)]),       
      row.names = NULL)

> myDat
    Subj group corr      y
 1    s1    g1 cor1  0.54966989
 2    s1    g1 cor2 -0.84160374
 3    s2    g1 cor1 -0.84160374
 4    s2    g1 cor2  0.03299794
 5    s3    g1 cor1  0.54966989
 6    s3    g1 cor2  0.03299794
 7    s4    g2 cor1  0.52414971
 8    s4    g2 cor2 -1.72760411
 9    s5    g2 cor1 -1.72760411
 10   s5    g2 cor2 -0.27786453
 11   s6    g2 cor1  0.52414971
 12   s6    g2 cor2 -0.27786453

It seems reasonable to assume compound symmetry for the variance-covariance matrix for the $N-1$ values of each subject. Therefore we can run the following linear mixed-effects modeling in R:

(fm <- lme(y~group, data=myDat, random=list(Subj=pdCompSymm(~0+corr))))

Linear mixed-effects model fit by REML
  Data: myDat 
  Log-restricted-likelihood: -14.06012
  Fixed: y ~ group 
(Intercept)     groupg2 
-0.08631197 -0.40746100 

Random effects:
 Formula: ~0 + corr | Subj
 Structure: Compound Symmetry
         StdDev    Corr  
corrcor1 0.7710823       
corrcor2 0.7710823 -0.378
Residual 0.2817136       

Number of Observations: 12
Number of Groups: 6 

> anova(fm)
            numDF denDF  F-value p-value
(Intercept)     1     6 2.246884  0.1845
group           1     4 1.108586  0.3518

My big concern about the above approach is that each value is used twice! On the other hand, the duplication is only for the purpose of estimating the variance-covariance structure, and it would have little (or no?) impact on the fixed effect of similarity comparison between the two groups. Is my argument sound enough?

  • $\begingroup$ Could you expand on what it would mean to "compare" two different correlation matrices of different sizes? What is the intended interpretation? $\endgroup$ – whuber May 6 '13 at 21:44
  • $\begingroup$ @whuber: We would like to see whether the two groups have significantly different correlation, which would be the fixed-effects part in the lme model: y ~ group. Does this clarify the situation? $\endgroup$ – bluepole May 7 '13 at 19:24
  • $\begingroup$ I'm sorry, it doesn't help me: could you be more specific about what a "different" correlation would be? Let's take a very simple example of two different-sized groups: a group of three and a group of two. For the group of three there are three correlation coefficients and for the group of two there is one correlation coefficient. Suppose, hypothetically, you had perfect information about these groups so there is no uncertainty about the values of the correlations. How would one go about determining whether these two groups have "different" correlations? $\endgroup$ – whuber May 7 '13 at 20:18
  • $\begingroup$ @whuber: Thanks for the example. I actually have 14 subjects in each group, leading to 91 unique correlation coefficients within each group. Does it make sense now to compare the "average" correlation between the two groups? Thanks again for your help! $\endgroup$ – bluepole May 7 '13 at 22:07
  • $\begingroup$ Compare how? If you mean just to compare the arithmetic mean of the $91$ coefficients, it's hard to see how that is meaningful at all. $\endgroup$ – whuber May 7 '13 at 22:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.