# Compare/adjust correlation coefficients for two groups of different sizes

I have a total of, say, 200 pairs of observations, but the split wasn't even between two conditions A and B. Suppose that condition A has 120 pairs of observations and B has 80. Now for both A and B, I can compute the correlation coefficients rA and rB, and I want to directly compare them/use them as predictors in regression. The question is: how do I know if rA and rB are comparable given the original number of observations was different? If that is an issue, is there a way I can adjust rA and rB to account for the difference, or account for that difference in regression?

Thanks a lot!

• What exactly do you mean by 'comparable'? Commented Sep 2, 2021 at 15:41
• @Bernhard I'm wondering if those two correlation measures can be 'directly' compared based on their values (rA>rB and therefore observations in condition A are more correlated) or 'indirectly' (rA>rB but the #of observations are different, so I'll need to account for that difference and adjust the correlation coefficients--but how?)
– hhh3
Commented Sep 2, 2021 at 20:24

Let's assume there is a large number of observations A and B which are correlated to some degree. A simulation for that in R might look like this:

library(ggplot2)
d <- MASS::mvrnorm(10000, mu = c(0,0), Sigma = matrix(c(1,.5,.5,1), ncol = 2))
d <- as.data.frame(d)
names(d)= c("A", "B")
ggplot(d) + geom_point(aes(x = A, y = B), alpha = .1)


Now we can draw 10 random pairs and compute a correlation coefficient as in

s <- sample.int(n = nrow(d), size = 10)
with(d, cor(A[s], B[s]))


Let's do that 30 times and see the different correlation coefficients we get:

> replicate(30, {s <- sample.int(n = nrow(d), size = 10)
+                     with(d, cor(A[s], B[s]))})
[1]  0.647630056  0.112336387  0.817311049  0.261255375
[5]  0.713635629  0.612139532  0.236262739  0.335451539
[9]  0.563006623  0.827905518  0.106554541  0.570146270
[13] -0.368941833  0.502980103  0.683218693  0.295538537
[17]  0.361098570  0.607926619 -0.112553317  0.335629279
[21]  0.832573073 -0.030073137  0.671726610  0.271553133
[25]  0.651124101  0.342336101  0.294655466  0.379537057
[29]  0.712296574  0.005328909


So these values vary largely but they vary around the true value .5 If we do the same with samples of n = 100 they vary less but arount the same true parameter:

> (replicate(30, {s <- sample.int(n = nrow(d), size = 100)
+                     with(d, cor(A[s], B[s]))}))
[1] 0.4732310 0.4247100 0.4376686 0.5357174 0.4073085 0.4909700
[7] 0.5987392 0.5019191 0.5867695 0.5013242 0.4978593 0.5127399
[13] 0.5239235 0.4497816 0.4727997 0.5187358 0.5231062 0.4616389
[19] 0.5348948 0.3857363 0.5776752 0.4042828 0.3589830 0.4108152
[25] 0.5650298 0.4457754 0.3523565 0.5041457 0.4808634 0.4939619


And if each sample is n = 1000 they still vary around the true value but the variation is less:

> (replicate(30, {s <- sample.int(n = nrow(d), size = 1000)
+                      with(d, cor(A[s], B[s]))}))
[1] 0.4703001 0.4977793 0.4324494 0.5093533 0.4490917 0.4933586
[7] 0.4397070 0.4791824 0.4903046 0.5145500 0.4950434 0.4556920
[13] 0.4655949 0.4877204 0.4562146 0.4747482 0.4962232 0.4533095
[19] 0.4867801 0.4749620 0.4646716 0.4827945 0.5335619 0.4675903
[25] 0.4843929 0.4921498 0.5251204 0.4569184 0.5097462 0.4499585


Let's visualize this:

n <- rep(c(10, 50, 100, 150, 200, 250, 300, 450, 600), each = 100)

samples <- data.frame(n = n,
r = sapply(n, \(n) {s <- sample.int(n = nrow(d), size = n)
return(with(d, cor(A[s], B[s])))
}))
ggplot(samples) +
geom_jitter(aes(x = n, y = r), width = 1, alpha = .5)


So independent on the sample size your observed correlation will be a correct estimate just the precision will increase with sample size. You can compare an r gained from n = 50 with an r gained from n = 5000. There are no corrections necessary, the only thing to keep in mind is the sampling error.

One way to deal with that would be confidence interval around your observed r or a statistical test. Steigers test might be what you want. I R the package psych has a function r.test for that.

• This makes a lot of sense. Thank you for the detailed explanation!
– hhh3
Commented Sep 3, 2021 at 13:57