Calculation for the Test of the Difference Between Two Independent Correlation Coefficients I have been told to take a closer look at: http://www.quantpsy.org/corrtest/corrtest.htm but how exactly does this work?
E.g. Given two groups of data A and B, each with one explanatory variable and one response variable, each with e.g. n=20 data points, I can calculate Spearman's Correlation Coefficient SCC yielding SCC = 0.9 in group A and in group B I get SCC=-0.9.
From the above, it is obvious that in group A the data is well correlated with a slope >0 and in group B also well correlated, but with a slope <0, so definitely a difference - Right?
Now to the real question: The SCC quantifies in a non-parametric way how well correlated the explanatory and response variables are, but I am not sure I understand how it works with testing the difference between two correlations? I am thinking that testing SCC=0.3 and SCC=0.8 should not yield anything, since basically one correlation is rather weak and the other is sort of ok? But the link yields p=0.02
 A: It sounds like you may be conflating the strength of correlation with the uncertainty about the correlation coefficient.
The test you're using essentially asks: how surprised would I be to observe these two correlation coefficients in my sample, if I already knew that, when I calculated the coefficients on the entire population, I would get the same answer for both?
When you calculate Spearman's coefficient on some data, you'll get a number that's somewhat different from the "true" correlation (the number you would get if you calculated the correlation on the entire population), because there's randomness in your sample. For instance, suppose I'm looking at the correlation between height and weight in my office, and I pick myself and my desk-mate. I'm taller and thinner than my desk-mate, so I calculate a correlation of -1 between height and weight in my sample--but I would be ill-advised to conclude that the population correlation was close to that! On the other hand, if I took a random sample of 100 people, and (hypothetically) observed a perfect negative correlation between height and weight in that sample, I would be a lot more confident that this result would generalize to the entire population.
The thing is, you can get a "kind of crappy" coefficient of 0.3, but if your sample is large enough, you can be very confident that the population correlation is actually around 0.3. It sounds like this is what happened to you: your sample size was large enough that, even though the effects don't look that different, you have enough precision to be fairly sure they aren't exactly the same.
