5
$\begingroup$

Calculating Pearson correlation using the cor.test() function in R, I found that the p-value does not necessarily correspond to the CI:

# reproducible example
# generate the data
set.seed(123)
N <- 200
X <- rnorm(N)
a <- 0.2
b <- 0.8
sigma <- 4
eps <- rnorm(N, 0, sigma)
Y <- a + b * X + eps
Y[1] <- 23.70932 # change a bit

cor.test(X, Y)

The resultant CI says the correlation is significant, while the p-value says otherwise:

t = 1.9717, df = 198, p-value = 0.05004
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 0.00002440203 0.27226422569
sample estimates:
      cor 
0.1387649 

The difference is small, but still quite large to be explained as a numerical error. This test does produce two-tailed test (for both the CI and the p-value) with the CI level of 0.95. Corresponding one-tailed test would be like this:

Y[1] <- 37.692
cor.test(X, Y, alternative = "greater")

with a similar problem. I tried to use the parameter exact = TRUE, but that's not related to the default calculation of Pearson correlation coefficient.

So, is this just a big imprecision (probably a bug, since the cor.test function doesn't provide any tolerance/epsilon parameter, which it should probably specify with this quite large imprecision?). Is the imprecision caused by not using an exact distribution of pearson coefficient, but just some approximations, that would yield slightly different results when calculating CI and p-value? Or is there some other principial problem?

$\endgroup$
10
$\begingroup$

Nearly all such questions are answered in the help on the relevant functions.

From the help for cor.test:

If method is "pearson", the test statistic is based on Pearson's product moment correlation coefficient cor(x, y) and follows a t distribution with length(x)-2 degrees of freedom if the samples follow independent normal distributions. If there are at least 4 complete pairs of observation, an asymptotic confidence interval is given based on Fisher's Z transform.

The CI based on Fisher's z will not exactly correspond to the result of the t-test.

The usual hypothesis tests rely on the distribution of the test statistic under the null. In this case, the null distribution for the test is based on the population correlation being zero. Under that circumstance, you get a t-test (under the assumptions), but when the population correlation is not zero you no longer have a t-distribution for the statistic, so the confidence interval isn't based on that. If an exact distribution were used for the CI (a calculation involving the hypergeometric function $_2F_1$, or a related series expansion), then it should correspond.

$\endgroup$
4
  • $\begingroup$ Hi @Glen_b, thanks for the answer! But, if the p-value of the $r$ can be calculated using t-test, this means that $r$ has t-distribution, doesn't it? And in that case, why isn't its CI calculated using t-distribution as well? $\endgroup$
    – Tomas
    Dec 1 '21 at 16:01
  • 2
    $\begingroup$ The null distribution for the test is based on the population correlation being zero. Under that circumstance, you get a t-test (under the assumptions), but when the population correlation is not zero you no longer have a t-distribution for the statistic, so the confidence interval isn't based on that. If an exact distribution were used for the CI, then it should correspond. $\endgroup$
    – Glen_b
    Dec 1 '21 at 16:09
  • 1
    $\begingroup$ @tdy My apologies; I see now that I shouldn't have rejected your edit. I was blind to the change of font in the html help and should have copied the text version of help instead. $\endgroup$
    – Glen_b
    Dec 1 '21 at 16:57
  • $\begingroup$ @Glen_b no worries, thanks for leaving a note $\endgroup$
    – tdy
    Dec 2 '21 at 1:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.