4
$\begingroup$

When dealing with ordinal correlations (e.g. Spearman's Rho, Kendall's Tau), one can non-parametrically test the null hypothesis of no correlation by a random permutation test (shuffling one of the two variables).

However, if we have a significantly non-zero correlation and we'd like to obtain a confidence interval for the correlation coefficient we need a standard error estimate that doesn't rely on the null hypothesis.

Bootstrapping seems to the way to go, but then comes the problem of duplicated observations. The bootstrapped samples contain duplicated observations due to resampling with replacement, and these duplicates generate ties that affect the ordinal correlations measures. For example, if one uses Kendall's Tau a, these spurious ties will decrease the correlation coefficient (i.e., introduce a negative bias).

How to deal with this problem? will a BCA (bias corrected and accelerated) confidence interval be appropriate here ? or alternatively, should we use a different resampling approach (e.g., jackknifing) instead?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

I found a paper that addresses your issue. The answer will depend on how correlated your data is. To summarise the paper, the Spearman correlation (r_s) is used to test the null hypothesis, but you are right to worry that we don't want the traditional Spearman test. The Fisher transform is z_r= arctanh(r_s). With this transform we can define a confidence interval = tanh[z_r±((N-3)^(-1/2)+z_CL)]

z_CL "is the percentile point of standard normal distribution below the which the subscripted portion of scores lies[sic]. For example, constructing a 95% CI for r=.5 and N=50 would proceed as follows: z_r = arctanh(.5) z_CL=z_.025=1.96, and CI=tanh(.5493 ± .1459 × 1.96)

To be honest, I don't understand the z_CL calculation, but if you want 95% then just use 1.96. Note that this method uses deviation from the theoretical "usual estimate" around Var(z_r)=(N-3)^(-1/2). It is possible that domain knowledge would suggest that this is not a good starting estimate.

I would try this initial estimate first, and if it does not give you reasonable CI then you can read on in the paper. I can also develop my answer more if need be.

In short, you may be able to avoid the bootstrapping issue.

In the case of making CI for Kendall's Tau there is already a cross validated question for this.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your answer. This indeed might be useful for Spearman. However, I'd like to know how should one generally approach this problem of bootstrapping ordinal correlations. In particular, I'd like to estimate confidence intervals to different variants of Kendall's Tau. $\endgroup$
    – Trisoloriansunscreen
    May 19, 2019 at 1:47
  • $\begingroup$ @Trisoloriansunscreen updated to reference a prior question on Kendall's Tau $\endgroup$
    – Czarking
    May 21, 2019 at 8:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.