I examined the relationship between two interval-scaled variables (x, y) in a small dataset (n=32). I used the Spearman's rank correlation coefficient since x is not normally distributed (evaluated with the Kolmogorov-Smirnoff-Test and by visual inspection of the histogram). I yielded r=-0.364 and p=0.040 which means a significant weak negative correlation. However, calculating the 95% confidence interval for rho (using packages "spearmanCI" and "boot" in R) results in [-0.740;0.002] and [-0.661, 0.069], respectively. Since zero is included in both of these intervals, the correlation could not be considered as "significant" in my opinion. Now I am irritated which ist the correct interpretation of this constellation and how it should be reported? Any hints would be appreciated. Thanks in advance!
1 Answer
I don't think there's a contradiction here. There are multiple possible 95% confidence intervals for $\rho$. $[-0.740, 0.002]$ is one, but there will also be an interval whose lower bound is -1 (the least possible correlation) and whose upper bound is less than zero.
An analogy with normal distributions (which I find easier to think about): suppose we know that $X \sim N(\mu, 1)$ and we are interested in investigating the null hypothesis that $\mu = 0$ versus the alternative hypothesis that $\mu > 0$. We have a single sample, $x_0 = 1.8$. Then the shortest possible 95% confidence interval for the value of $\mu$ would have bounds $1.8 \pm \Phi^{-1}(0.975)$ i.e. $1.8 \pm 1.96$, which contains $0$. Yet the p-value of our result is $0.036$, which is less than the threshold of $0.05%$. This is because the confidence interval that corresponds to our hypothesis test is actually $(1.8 - \Phi^{-1}(0.95), \infty) = (1.8 - 1.645, \infty)$.
spearmanCI, butbootcertainly uses Monte Carlo methods to estimate the CI, so the results are a little noisy, and won't necessarily be the same each time you run them. $\endgroup$