5
$\begingroup$

My setting looks like that I have several subjects $i$ (let's say 5) and for each subject I measure the lists/vectors $A_i$, $B_i$, $C_i$ and $D_i$. Then, for each subject $i$, I calculate the correlation between $A_i$ and $B_i$ (let me call this correlation $AB_i$) as well as between $C_i$ and $D_i$ ($CD_i$). Now, I have for each subject $i$ one correlation coefficient $AB$ and $CD$. Thus, for e.g., five subjects, I have five $AB_i$ correlation coefficients such as $AB'=[AB_1=-0.1, AB_2=0.2, AB_3=0.25, AB_4=0.3, AB_5=0.2]$ and five $CD_i$ coefficients such as $CD'=[CD_1 = 0.8, CD_2=0.8, CD_3=0.75, CD_4=0.9, CD_5=0.7]$.

Now, I want to test the null hypothesis that the correlation coefficients $AB'$ are similar to $CD'$. So basically, whether the mean of $AB'$ is the same as the mean of $CD'$.

Note that $A_i$, $B_i$, $C_i$ and $D_i$ are not repeated measures and focus on different things. Actually, $A_i$ and $C_i$ as well as $B_i$ and $D_i$ correspond to the same variable being measured, but for different sub-groups of my subjects. So, a subject can correspond to a specific event where I can distinguish between sub-groups. E.g., suppose that you study five different school classes (subjects) and for each class you distinguish between males and females (sub-groups). Then you measure two variables (vectors) for both males ($A_i$, $B_i$) and females ($C_i$, $D_i$). Now, I want to know whether $A_i$ correlates similarly to $B_i$ as $C_i$ correlates to $D_i$ across all subjects (school classes).

I know how to calculate a statistical significance test regarding the differences between single correlation coefficients for individual subjects. There are several approaches available; one is to do a Fisher z transformation and then determine the z statistic with known standard deviation. However, how can I do this across subjects (for the mean) in one step?

Normally, I would just pick a t-test. However, the issue I see here is that the correlation coefficients are not normally distributed which is why we could again work with the Fisher transformation somehow. I am unsure how though.

Maybe average the transformed correlations and then conduct a z-test?

I have studied some similar questions withouth finding any answer to this problem. Hope someone has an idea of how to approach this.

Improve this question
$\endgroup$
15
  • $\begingroup$ Could you please explain what you mean by "both correlation coefficients are equal across subjects"? You seem to have twice as many computed correlation coefficients as subjects. It's hard to make sense of "both" in that case. $\endgroup$ – whuber♦ Jul 9 '15 at 20:09
  • $\begingroup$ Suppose we have 10 subjects. For each subject I measure a vector A,B,C and D. For each subject I calculate the correlation coefficient between A and B as well as B and C leading to the coefficients AB and CD. Now, for each subject, I can do a statistical significance test whether both independent coefficients are equal (e.g., with the Fisher test). However, I want to do this across subjects. $\endgroup$ – fsociety Jul 9 '15 at 20:26
  • $\begingroup$ That helps, but I'm still totally lost concerning what you mean by a test "across subjects," because I can think of many distinctly different things this might mean. What is your null hypothesis? What is the alternative? $\endgroup$ – whuber♦ Jul 9 '15 at 21:02
  • $\begingroup$ My null hypothesis is that the correlations between AB and CD are equal. I calculate AB and CD multiple times for multiple subjects. So my vectors for let's say 5 subjects might look like $AB=[0.3,0.2,0.25,-0.1]$ and $CD=[0.7,0.8,0.8,0.75]$. $\endgroup$ – fsociety Jul 9 '15 at 21:06
  • 1
    $\begingroup$ Damn, it is late, sorry. There should be 5 elements in the vectors (can't edit it now). The elements of the vectors are correlation coefficients though that are already calculated for some other vectors. So basically I have vectors of correlation coefficients and I want to test the null that those are similar. $\endgroup$ – fsociety Jul 9 '15 at 21:15
2
+50
$\begingroup$

Short answer: Yes, transform each correlation ($AB_i$ and $CD_i$) using the Fisher $r-to-z'$ transform: $f(r)=\frac12 ln \frac{1+r}{1-r}$. Then, perform an independent samples t-test to test the null hypothesis of $\mu_{f(AB)}=\mu_{f(CD)}$.

Rationale: You're right that approximate normality is important here. With only a small number of subjects, you can't count on the central limit theorem to address the (often) non-normal sampling distribution of r. The sampling distribution of r will only be approximately normal when $\rho$ is close to 0 or when n is very large (here, I refer to the n used to compute the correlation, not the n indexed by i in your question). Based on the examples in your question, I'm guessing n is modest, and if you already knew the $\rho$'s, there'd be no point in asking your question. Bottom line: the Fisher $r-to-z'$ will probably help here.

It sounds like you're comparing independent samples, especially based on this part of your description:

$A_i$ and $C_i$ as well as $B_i$ and $D_i$ correspond to the same variable being measured, but for different sub-groups of my subjects.

So, it makes sense to use an independent samples t-test.

Improve this answer
$\endgroup$
4
  • $\begingroup$ From late entries in the comment thread above, this might not be a paired situation. Will your approach also work if these are unpaired comparisons? $\endgroup$ – EdM Jul 14 '15 at 13:50
  • $\begingroup$ Thanks! I have extended my original post to adhere to the comments by @EdM. $\endgroup$ – fsociety Jul 14 '15 at 14:36
  • $\begingroup$ Thank you, that answer confirm my personal intuition. One final thing though: Instead of a t-test, could I also use a z-test here as the Fisher transformation has known variance? $\endgroup$ – fsociety Jul 16 '15 at 11:16
  • 1
    $\begingroup$ Good point. I suppose you could do a z-test if you analytically solve for the variance of $mean(f(AB))-mean(f(CD))$. $\endgroup$ – Anthony Jul 16 '15 at 11:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.