How to calculate a confidence interval for Spearman's rank correlation?

Question

Wikipedia has a Fisher transform of the Spearman rank correlation to an approximate z-score. Perhaps that z-score is the difference from null hypothesis (rank correlation 0)?

This page has the following example:

4, 10, 3, 1, 9, 2, 6, 7, 8, 5
5, 8, 6, 2, 10, 3, 9, 4, 7, 1
rank correlation 0.684848
"95% CI for rho (Fisher's z transformed)= 0.097085 to 0.918443"

How do they use the Fisher transform to get the 95% confidence interval?

Answer 1

In a nutshell, a 95% confidence interval is given by
$$\tanh(\operatorname{arctanh}r\pm1.96/\sqrt{n-3}),$$ where $r$ is the estimate of the correlation and $n$ is the sample size.

Explanation: The Fisher transformation is arctanh. On the transformed scale, the sampling distribution of the estimate is approximately normal, so a 95% CI is found by taking the transformed estimate and adding and subtracting 1.96 times its standard error. The standard error is (approximately) $1/\sqrt{n-3}$.

EDIT: The example above in Python:

import math
r = 0.684848
num = 10
stderr = 1.0 / math.sqrt(num - 3)
delta = 1.96 * stderr
lower = math.tanh(math.atanh(r) - delta)
upper = math.tanh(math.atanh(r) + delta)
print "lower %.6f upper %.6f" % (lower, upper)

gives

lower 0.097071 upper 0.918445

which agrees with your example to 4 decimal places.

Answer 2

Maybe some additional remarks about the comment of @chl

The Spearman correlation can be seen as a Pearson correlation of the ranks. Ranks clearly do not follow a normal distribution, with the consequence that the variance of the Fisher transformation ($\zeta$) is not well approximated by $(n-3)^{-1}$ especially at large absolute values of $\rho_s$ and low number of observations. Various empirically motivated adjustments of the variance have been suggested in literature. They are compared in Bonnett and Wright (2000), including the one with the 1.06 factor also mentioned in Wikipedia. Bonnett and Wright (2000) finally recommended the following variance estimator

$$ \sigma^2_\zeta = \frac{1 + r_s^2/2}{n-3} $$

where $r^2_s$ is the sample Spearman correlation and $n$ is the number of observations. This leads to the following $(1-\alpha)$-CI

$$ \tanh\left(\text{arctanh}(r_s) \pm \sqrt{\frac{1 + r_s^2/2}{n-3}} z_{\frac{\alpha}{2}}\right). $$

where $z_{\frac{\alpha}{2}}$ is the $\frac{\alpha}{2}$-quantile of the standard normal distribution. In R, this function would calculate the CI

spearman_CI <- function(x, y, alpha = 0.05){
  rs <- cor(x, y, method = "spearman", use = "complete.obs")
  n <- sum(complete.cases(x, y))
  sort(tanh(atanh(rs) + c(-1,1)*sqrt((1+rs^2/2)/(n-3))*qnorm(p = alpha/2)))
}

Ruscio (2008) further suggests to replace the normal quantile $z_{\frac{\alpha}{2}}$ by a $t$-quantile with $n-2$ degrees of freedom in order to get better coverage.

Still, the CI is approximate. Especially in situations where

• $\rho_s > 0.95$ (where $\rho_s$ is the true population Spearman correlation)
• $n < 25$
• ordinal data

a bootstrap CI has clearly better properties (Ruscio 2008, Bishara and Hittner 2017).

Source

• Bishara, Anthony J., and James B. Hittner. “Confidence Intervals for Correlations When Data Are Not Normal.” Behavior Research Methods 49, no. 1 (February 1, 2017): 294–309. https://doi.org/10.3758/s13428-016-0702-8.
• Bonett, Douglas G., and Thomas A. Wright. “Sample Size Requirements for Estimating Pearson, Kendall and Spearman Correlations.” Psychometrika 65, no. 1 (March 1, 2000): 23–28. https://doi.org/10.1007/BF02294183.
• Ruscio, John. “Constructing Confidence Intervals for Spearman’s Rank Correlation with Ordinal Data: A Simulation Study Comparing Analytic and Bootstrap Methods.” Journal of Modern Applied Statistical Methods 7, no. 2 (November 1, 2008). https://doi.org/10.22237/jmasm/1225512360.