2
$\begingroup$

I have two questions with respect to comparing two correlation coefficients.

Cohen (1977) suggested to use Cohen's q to compare two Pearson's correlation coefficients by first transforming r with Fisher's zr transformation into z values.

Cohen's q = z1 - z2.

The estimated variance for Cohen's q is calculated as follows (see here): 1/(N1-3)+1/(N2-3)

Question 1: I would like to calculate the 95% CI for Cohen's q but am unsure of how to do this. Could anyone help me with this please?

Question 2: Is it okay to use the same procedure with Spearman's correlation coefficients (i.e. Fisher's zr transformation and then calculate the difference)?


Update:

I have just seen that the R package cocor includes Zou's (2007) confidence interval of the difference between two correlation coefficients. Obviously, this might be helpful as some sort of workaround. If the CI does not include 0, the difference is significant, and at the same time it gives an estimate of the CI of this difference. Cocor can also be used to test whether two correlation coefficients are significantly different from each other based on Fisher's zr transformation. More information can be found here.

However I would still be highly interested in any suggestions/answers concerning the two questions above.

$\endgroup$
  • $\begingroup$ Does this mean to say that CIs for Cohen's d and Cohen's q are calculated in exactly the same way? $\endgroup$ – grey Feb 19 '16 at 9:55
  • $\begingroup$ My apologies, I misread that. $\endgroup$ – COOLSerdash Feb 19 '16 at 10:34
  • 1
    $\begingroup$ Q1: I'm not aware of any closed-form formulae for the 95%-CI (but I didn't perform a thorough search). I'd advise you to just use a nonparametric bootstrap. $\endgroup$ – COOLSerdash Feb 28 '16 at 16:22
  • $\begingroup$ Thanks for your suggestion. If you happen to have any links for starting points of further reading, this would be most appreciated. $\endgroup$ – grey Feb 29 '16 at 13:46
3
$\begingroup$

I think I have found out how to calculate the 95% CI for Cohen's q. The approach is based on the fact that Cohen's q is a z-score because of the r-to-z transformation.

Cohen's $q=z_1-z_2$

$\mathrm{SE}(q)=\sqrt{\dfrac{1}{N_1-3}+\dfrac{1}{N_2-3}}$

For 95% CI: z = 1.96

Upper and lower 95% CI for Cohen's q is calculated as

$q\pm 1.96\cdot \mathrm{SE}(q)$

$\endgroup$
2
$\begingroup$

I would suggest bootstrapping. Here is a quick and dirty implementation in R. I first simulate some values from a multivariate normal distribution. In this case, both samples have the same size, which may not be the case in real-world examples.

In order to understand how bootstrapping works in general, read this post. The confidence interval is based on taking $\alpha/2$ and $1-\alpha/2$ percentiles of the bootstrap sample.

# Load packages
library(MASS)

# Simulate some data
cov.mat <- matrix(c(
  5, 0.75*sqrt(5)*sqrt(6), 0, 0,
  0.75*sqrt(5)*sqrt(6), 6, 0, 0,
  0, 0, 7, 0.9*sqrt(7)*sqrt(15),
  0, 0, 0.9*sqrt(7)*sqrt(15), 15
), 4, 4, byrow = TRUE)

set.seed(17)
dat <- mvrnorm(n = 200, mu = c(100, 150, 50, 75), cov.mat, tol = 1e-6)

# Check correlation coefficients
cor(dat)

cohens.q <- NULL # To store Cohen's q

N.reps <- 10000 # Number of bootstrap-repetitions

N12 <- length(dat[, 1]) # Sample size 1
N34 <- length(dat[, 3]) # Sample size 2

for(i in 1:N.reps) {

  tmp1 <- dat[sample(1:N12, size = N12, replace = TRUE), c(1, 2)]
  tmp2 <- dat[sample(1:N34, size = N34, replace = TRUE), c(3, 4)]

  cor12 <- cor(tmp1[, 1], tmp1[, 2])
  cor34 <- cor(tmp2[, 1], tmp2[, 2])

  z1 <- atanh(cor12)
  z2 <- atanh(cor34)

  cohens.q[i] <- z1 - z2

}

hist(cohens.q, breaks = "Freedman-Diaconis", col = "steelblue")
qqnorm(cohens.q)
qqline(cohens.q)

mean(cohens.q)
sd(cohens.q)
median(cohens.q)

# Confidence intervals based on percentiles
alpha <- 0.05
quantile(cohens.q, c(alpha/2, 1-alpha/2))

      2.5%      97.5% 
-0.9856246 -0.5628550
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.