6
$\begingroup$

Since a few days I do not get ahead when trying to compare two Pearson correlation coefficients. Imagine that I've got two datasets where on each I do a correlation between Land Surface Temperature and an urban metric. The datasets are different in their length, so the first one has round about 160.000 observables and the second one has about 2400 observables. For the correlation on the first dataset I get a Pearson of -0.74 and for the second I get -0.885. Now I want to find out whether these coefficients are significantly different from each other. Is there any appropriate method you could suggest?

I already played around with the Fisher-Z-Transformation, but from my point of view with no purposeful results. When I calculate the Fisher-Z on my coefficients, it results in 20.95 (Fisher-Z).

I would be very happy about suitable advices.

Improve this question
$\endgroup$
4
  • 6
    $\begingroup$ I have to suggest that this is usually the wrong question, which is a good reason why discussions of such comparisons are a little hard to find. The same correlation could be consistent with quite different relationships and conversely different correlations could be consistent with the same relationship. The scientific issue is presumably whether the two groups show the same relationships and some kind of regression model for data jointly and separately is then usually the way forward. $\endgroup$
    – Nick Cox
    Apr 9, 2015 at 13:00
  • 2
    $\begingroup$ There are many issues with comparing these data that will render invalid any answer you get: the surface temperatures will exhibit strong spatial correlation; the urban metric is likely some combination of categorical variables that, at a minimum, would need to be re-expressed in a nonlinear fashion; the spatial supports of the data are not directly comparable; and, as @Nick points out, comparing correlation coefficients tells you little of relevance. I would like to suggest that you instead ask a question more relevant to your study objective and that you describe your data in more detail. $\endgroup$
    – whuber ♦
    Apr 9, 2015 at 14:05
  • $\begingroup$ I somehow missed your post @whuber. To bring some light in the dark, I´ll describe my data more detailed: I´ve got a spatial dataset and calculated urban metrics within 30 by 30 meter grid cells. So I got 160.000 grid cells and each of them has one value for land surface temp and one value for an urban metric. I calculated the correlation between them. Then I computed the urban metrics on 60 by 60 m cells and did the correlation again . Now I wanted to check, in a statistical way, whether the correlations are significantly different from another. $\endgroup$
    – Axel.Foley
    Apr 9, 2015 at 14:34
  • 3
    $\begingroup$ As @whuber suggested, you really need to ask a new question (meaning a new thread). But in this case, you are averaging cells in groups of 4, presumably, so the original and reduced datasets are dependent on each other. However, 159200/4 is far more than 2400. More to the point, it is expectable that correlations based on averaging are different, as widely discussed in spatial statistics. $\endgroup$
    – Nick Cox
    Apr 9, 2015 at 14:45

2 Answers 2

Reset to default
10
$\begingroup$

There are various tests you can apply. Biedenhofen & Musch (2015, PLoS ONE) give pointers and describe the cocor package for R, which implements these tests. You can also submit your correlations for testing to a web tool which internally uses the cocor package.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The cocor package seems to be a handy tool. I ran the cocor package with my parameters via the web tool as you suggested. The output of that calculation is the following: Comparison between r1.jk = -0.747 and r2.hm = -0.885 Difference: r1.jk - r2.hm = 0.138 Group sizes: n1 = 159200, n2 = 2400 Null hypothesis: r1.jk is equal to r2.hm Alternative hypothesis: r1.jk is not equal to r2.hm (two-sided) Alpha: 0.05 fisher1925: Fisher's z (1925) z = 21.0047, p-value = 0.0000 Null hypothesis rejected $\endgroup$
    – Axel.Foley
    Apr 9, 2015 at 13:21
0
$\begingroup$

The cocor package seems to be a handy tool. I ran the cocor package with my parameters via the web tool as you suggested. The output of that calculation is the following:

Comparison between r1.jk = -0.747 and r2.hm = -0.885

Difference: r1.jk - r2.hm = 0.138

Group sizes: n1 = 159200, n2 = 2400

Null hypothesis: r1.jk is equal to r2.hm

Alternative hypothesis: r1.jk is not equal to r2.hm (two-sided) Alpha: 0.05

fisher1925: Fisher's z (1925)

z = 21.0047, p-value = 0.0000

Null hypothesis rejected

This seems pretty promising to me, but how do I have to interpret the result? The correlations are obviously different from another, but is the difference significant?

Improve this answer
$\endgroup$
3
  • 4
    $\begingroup$ That's what the test is saying. But note: (1) the calculation does nothing to adjust for spatial dependence; (2) with sample sizes this large, almost any difference qualifies as significant. I get 95% confidence intervals for $r = -0.787, n = 159200$ as $(-0.749, -0.745)$ and for $r = -0.885, n = 2400$ as $(-0.893, -0.876)$, underlining that an explicit test of the difference is hardly needed. (Spatial dependence is a more complicated problem, but a side-issue.) As so often in statistical science, a test of strong effects with large sample size doesn't itself add insight. $\endgroup$
    – Nick Cox
    Apr 9, 2015 at 13:55
  • $\begingroup$ So thanks for mentioning the spatial issue, but that makes the result less promising and relieble to me since I´m working with data, which is based on spatial datasets. Now, when dealing with sample sizes that large, it seems not to be an appropriate test to analyse significant differences between correlations, isn´t it? $\endgroup$
    – Axel.Foley
    Apr 9, 2015 at 14:12
  • 2
    $\begingroup$ I don't understand whether you are asking another question or the same question differently put. My view, echoed by @whuber, is that the significance calculation is strictly invalid without adjustment for spatial dependence, but the larger issue is very simple: the calculation is not useful. $\endgroup$
    – Nick Cox
    Apr 9, 2015 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.