# What is the best way to calculate confidence interval for spearman correlation by bootstrapping? [closed]

I find important differences for the confidence interval values between the two methods below with bootstrap :

First : quantile(Rs, prob=c(0.025, 0.975))

and Second : tanh(atanh(R) ± 1.96 sqrt(n-3)) (second explained [there]How to calculate a confidence interval for Spearman's rank correlation? )

Is there a mistake or why these differences ? Have a nice day.

Correlate=c()
nb=1000

for ( i in 1:nb)
{
A=rnorm(500, 10,3)
E=runif(500, min=0, max=12)
B=A*2+E*2+E
Z=cor(A,B, method="spearman")
Correlate=append(Correlate, Z)
if (i==1) origin=Z
}

#quantile
print(quantile(Correlate, prob=c(0.025, 0.975)))

#cal
stderr=1/sqrt(500-3) #edit not NB
delta=1.96*stderr
up=tanh(atanh(origin) + delta) #edit
down=tanh(atanh(origin) - delta) #edit
print(up)
print(down)


Edit: for quantile see [there]https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/quantile

quantile is used to find the values of the percentile 97.5 and 2.5, see https://en.wikipedia.org/wiki/97.5th_percentile_point

Results:

2.5%     97.5%
0.3899030 0.5288183
print(up)
[1] 0.4804465
print(down)
[1] 0.3794499


Edit thanks Mathemagician777 for comment on delta

• Hi! Welcome to Stack Exchange. Can you please clarify your code a bit and maybe explain in words/text what happens in short rather than giving functions like quantile(...)? I don't see the variable 'delta' used in your code for example and also don't see the variable 'origin' defined somewhere. Commented Feb 15 at 7:24

There are a number of issues with your approach, not just with the confidence interval itself:

1. A bootstrap is usually performed by resampling an observed sample. You draw new samples from a population each time. This is akin to replicating your experiment as a whole, or perhaps an m-of-n bootstrap on a much bigger sample, but it's not how you would (or often could) use a bootstrap in practice.
2. You include the original sample statistic in your bootstrap distribution. The impact of this will likely become smaller as the number of resamples increases, but it's not something you're supposed to do.
3. The big one: you use the number of bootstrap resamples in the calculation of your confidence interval. This is something you should never do: the bootstrap variance cannot depend on the number of resamples you take other than through the sampling variability that this causes, because the number of resamples is arbitrary. Your calculation will produce a CI width towards zero as the number of resamples goes towards infinity. The $$n$$ in such formulae should be the number of original samples or the size of the resample unless you're specifically doing calculations involving the number of bootstrap replicates. The only reason why your interval isn't more off is that your number of original samples (500) is relatively close to your number of resamples (1,000).

The last problem leads to another issue: the only quantities in the hyperbolic arc-tangent confidence interval formula are the original sample size and the original sample correlation statistic - nothing is actually there to bootstrap/resample. A bootstrap approximation to the standard error would be the standard deviation of your bootstrap distribution.

Bringing all of this together I would approach like so:

nb=1000
Correlate <- rep(NA, nb)
set.seed(1)

## Create ORIGINAL sample
A <- rnorm(500, 10,3)
E <- runif(500, min=0, max=12)
B <- A*2+E*2+E
origin <- cor(A,B, method="spearman")

## Bootstrap the original sample pairs
for (i in 1:nb) {
s <- sample(seq_along(A), replace=TRUE)
Correlate[i] <- cor(A[s], B[s], method="spearman")
}

## Raw bootstrap quantiles
quantile(Correlate, probs = c(.025, .975))
#>   2.5%  97.5%
#> 0.4519 0.5668

## Normal approximation via SE = SD(bootstrap)
origin + c(-1, 1) * qnorm(.975) * sd(Correlate)
#> 0.4522 0.5744

## atanh transform of the same
tanh(atanh(origin) + c(-1, 1) * qnorm(.975) * sd(Correlate))
#> 0.4670 0.5569

## For good measure: the original atanh interval (not resampled)
tanh(atanh(origin) + c(-1, 1) * qnorm(.975) * 1/sqrt(length(A)-3))
#> 0.4457 0.5751


Like so you see that the raw quantiles and the normal approximation bootstrap intervals agree relatively well in this case.

• Thanks a lot for your complete answer. It is very clear I vote for you. Cross validated said post is off topic and suggest to move or delete. Commented Feb 15 at 14:19

I'm not sure if you took it into account based on only the code you provided, but note that when bootstrapping for a CI for correlation, you have to bootstrap the dependence between the 2 variables as well! If you take independent bootstrap samples (which I think is what happened in the first part of your code), you will not get valid results.

A last small note: when performing bootstrap in R, R has some very nice package and functions for this. It is worth checking out the boot package for this!