I have data for two classes A and B for N participants (same group -> dependent) in an experiment. Both classes are described by a feature vector. For each participant i have a certain number of samples (=feature vectors) for each class.

Now, in a first step I am interested in the difference of the average within-class sample-to-sample correlation of A (that is, the sample-to-sample reliability of A) and the average between-class sample-to-sample correlation (r_AB; that is, the similarity of A and B) for each participant separately. I could do the same for B, but for simplicity I focus on A. Thus I subtract z-transformed r_AB (=z_AB) from z-transformed r_A (=z_A) for each participant:

diffz_A = z_A - z_AB

(let's suppose diffz_A has now N values, i.e. one z-transformed correlation difference for each participant)

Now I want to look at two time-points 1 and 2, such that i have two instances of those correlation differences:

diffz_A1 and diffz_A2

How can I test whether the two correlation differences diffz_A1 and diffz_A2 are significantly different?
Upon literature consultation I found that there is a specific test for testing the difference of two dependent non-overlapping correlations (Steiger et al. 1980). However, it's absolutely unclear how to apply this test to differences of correlations, which in turn are partly based on within-class correlations on the participant level..

So my questions are:

  • do i need a complicated z-test in first place for testing between two differences of correlations? Or can I use a simple t-test?

  • If no, which test should i use and how?

  • and somewhat unrelated: if i want to transform a correlation difference back from z-space to r-space (using tanh), can i apply tanh simply to the obtained correlation difference in z-space (i.e. meandiffr = tanh(diffz_A)) or do i have to apply tanh separately to the two terms in the subtraction formulae?

  • 1
    $\begingroup$ I wrote a tutorial review centred on Fisher's z transformation for correlations published as stata-journal.com/sjpdf.html?articlenum=pr0041 Included are some worked examples of how to look at differences of correlations and several references to textbook discussions of the same. (Incidentally, I found very few really good treatments in the literature, and all too many that were too brief or too poorly explained to be of much use to anyone.) My paper is Stata-based, but translation to any decent statistical language should be trivial. $\endgroup$ – Nick Cox Dec 4 '13 at 18:25
  • $\begingroup$ Thanks for your reply @NickCox - however i suppose you're referring to tests assessing the difference of correlations, whereas i am interested in testing the difference of correlation differences. But maybe that's the same thing afterall... $\endgroup$ – Matthias Dec 4 '13 at 18:50
  • $\begingroup$ I think you need to spell out what your model is for the distribution of whatever you are focusing on. I don't have concrete suggestions for how to handle that, but just a visceral feeling that differences in correlations are tricky enough and differences between differences a small nightmare. $\endgroup$ – Nick Cox Dec 4 '13 at 18:59
  • $\begingroup$ Hmm, not sure about the model for the distribution. However, it might be important, that in my case i have correlation values* for each participant in both conditions as opposed to a single between-subject correlation for each condition (*well, correlation differences, to be precise). $\endgroup$ – Matthias Dec 4 '13 at 19:09
  • $\begingroup$ An optimistic possibility is that your correlations are, in effect, data which you can feed into (e.g.) an analysis of variance (and that here as elsewhere behaviour might be simpler on the z scale). $\endgroup$ – Nick Cox Dec 4 '13 at 19:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.