How to calculate 95% CI for Cohen's q?

Question

I have two questions with respect to comparing two correlation coefficients.

Cohen (1977) suggested to use Cohen's q to compare two Pearson's correlation coefficients by first transforming r with Fisher's z_r transformation into z values.

Cohen's q = z₁ - z₂.

The estimated variance for Cohen's q is calculated as follows (see here): 1/(N1-3)+1/(N2-3)

Question 1: I would like to calculate the 95% CI for Cohen's q but am unsure of how to do this. Could anyone help me with this please?

Question 2: Is it okay to use the same procedure with Spearman's correlation coefficients (i.e. Fisher's z_r transformation and then calculate the difference)?

---

Update:

I have just seen that the R package cocor includes Zou's (2007) confidence interval of the difference between two correlation coefficients. Obviously, this might be helpful as some sort of workaround. If the CI does not include 0, the difference is significant, and at the same time it gives an estimate of the CI of this difference. Cocor can also be used to test whether two correlation coefficients are significantly different from each other based on Fisher's z_r transformation. More information can be found here.

However I would still be highly interested in any suggestions/answers concerning the two questions above.

Answer 1

I think I have found out how to calculate the 95% CI for Cohen's q. The approach is based on the fact that Cohen's q is a z-score because of the r-to-z transformation.

Cohen's $q=z_1-z_2$

$\mathrm{SE}(q)=\sqrt{\dfrac{1}{N_1-3}+\dfrac{1}{N_2-3}}$

For 95% CI: z = 1.96

Upper and lower 95% CI for Cohen's q is calculated as

$q\pm 1.96\cdot \mathrm{SE}(q)$

Answer 2

I would suggest bootstrapping. Here is a quick and dirty implementation in R. I first simulate some values from a multivariate normal distribution. In this case, both samples have the same size, which may not be the case in real-world examples.

In order to understand how bootstrapping works in general, read this post. The confidence interval is based on taking $\alpha/2$ and $1-\alpha/2$ percentiles of the bootstrap sample.

# Load packages
library(MASS)

# Simulate some data
cov.mat <- matrix(c(
  5, 0.75*sqrt(5)*sqrt(6), 0, 0,
  0.75*sqrt(5)*sqrt(6), 6, 0, 0,
  0, 0, 7, 0.9*sqrt(7)*sqrt(15),
  0, 0, 0.9*sqrt(7)*sqrt(15), 15
), 4, 4, byrow = TRUE)

set.seed(17)
dat <- mvrnorm(n = 200, mu = c(100, 150, 50, 75), cov.mat, tol = 1e-6)

# Check correlation coefficients
cor(dat)

cohens.q <- NULL # To store Cohen's q

N.reps <- 10000 # Number of bootstrap-repetitions

N12 <- length(dat[, 1]) # Sample size 1
N34 <- length(dat[, 3]) # Sample size 2

for(i in 1:N.reps) {

  tmp1 <- dat[sample(1:N12, size = N12, replace = TRUE), c(1, 2)]
  tmp2 <- dat[sample(1:N34, size = N34, replace = TRUE), c(3, 4)]

  cor12 <- cor(tmp1[, 1], tmp1[, 2])
  cor34 <- cor(tmp2[, 1], tmp2[, 2])

  z1 <- atanh(cor12)
  z2 <- atanh(cor34)

  cohens.q[i] <- z1 - z2

}

hist(cohens.q, breaks = "Freedman-Diaconis", col = "steelblue")
qqnorm(cohens.q)
qqline(cohens.q)

mean(cohens.q)
sd(cohens.q)
median(cohens.q)

# Confidence intervals based on percentiles
alpha <- 0.05
quantile(cohens.q, c(alpha/2, 1-alpha/2))

      2.5%      97.5% 
-0.9856246 -0.5628550