6
$\begingroup$

I am working my way through (T.J. Diccio & B. Efron, "Bootstrap Confidence Intervals", Statistical Science, 1996, 11(3), 189–228), and I'm stuck even before I get to the good stuff.

In the introduction, there is a table of before/after treatment values called cd4, and the correlation $\hat\theta$ is given as 0.723. The paper then states, "We can find an exact interval for $\theta$ if we are willing to assume bivariate normality ... The exact central 90% interval is (0.47, 0.86)."

Here I stipulate that I am a stats moron. When I key in the data and compute a confidence interval by hand in R, either using the Fisher transform or the T distribution, I don't get that interval. How does one compute the interval they are talking about?

> cd4
   subj base oneyr
1     1 2.12  2.47
2     2 4.35  4.61
3     3 3.39  5.26
4     4 2.51  3.02
5     5 4.04  6.36
6     6 5.10  5.93
7     7 3.77  3.93
8     8 3.35  4.09
9     9 4.10  4.88
10   10 3.35  3.81
11   11 4.15  4.74
12   12 3.56  3.29
13   13 3.39  5.55
14   14 1.88  2.82
15   15 2.56  4.23
16   16 2.96  3.23
17   17 2.49  2.56
18   18 3.03  4.31
19   19 2.66  4.37
20   20 3.00  2.40
> r = cor(cd4$base, cd4$oneyr)
> r
[1] 0.7231654

Fisher transform:

> se = 1/sqrt(17)
> se
[1] 0.2425356
> tanh(c(r-1.6448*se, r+1.6448*se))
[1] 0.3133382 0.8082940

T distribution:

> sr = sqrt((1-r^2)/(20-2))
> sr
[1] 0.1627936
> tc = abs(qt(0.05/2, 18))
> c(r-tc*sr, r+tc*sr)
[1] 0.3811486 1.0651821
$\endgroup$
3
  • $\begingroup$ This is the approximate interval based on Fisher's Z, it is not exact. $\endgroup$ Jul 29, 2014 at 17:00
  • $\begingroup$ From Monteverde's answer below, I did the Fisher transform wrong, it should be 'z=atanh(r); tanh(c(zp-1.6448*se, zp+1.6448*se))'. $\endgroup$ Jul 29, 2014 at 17:07
  • 1
    $\begingroup$ Yes, I mean the claim in the paper is wrong: this is not an exact CI. $\endgroup$ Jul 29, 2014 at 17:10

2 Answers 2

9
$\begingroup$

You can get the same values with:

cor.test(cd4$baseline, cd4$oneyear, method = "pearson", conf.level = 0.9)

The method used to obtain such interval is explained in:

http://stat.ethz.ch/R-manual/R-patched/library/stats/html/cor.test.html

If method is "pearson", the test statistic is based on Pearson's product moment correlation coefficient cor(x, y) and follows a t distribution with length(x)-2 degrees of freedom if the samples follow independent normal distributions. If there are at least 4 complete pairs of observation, an asymptotic confidence interval is given based on Fisher's Z transform.

So, you can implement your own code to obtain C.I. by following these instructions if you wish to do so. See also:

http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#Testing_using_Student.27s_t-distribution

$\endgroup$
1
  • $\begingroup$ Thank you. Walking through the source to cor.test with my data helped me see I was screwing up the Fisher's Z transform. $\endgroup$ Jul 29, 2014 at 17:09
2
$\begingroup$

You already mentioned that the Fisher transformation is not correct in your code. You first have to transform r to a z value (atanh part), then you add and subtract the standard error with the appropriate multiplier to get the correct confidence (as you did correctly). Finally, you have to transform the whole thing back into the r-metric (tanh part).

se <- 1/sqrt(17)
r <- 0.7231654
tanh(atanh(r)+c(1,-1)*qnorm(.95)*se)

Which results in

[1] 0.8650790 0.4740748

As mentioned in the comments, this is NOT the exact interval! To find an exact interval check out this work by Shieh: http://link.springer.com/10.1007/s11336-04-1221-6

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.