8
$\begingroup$

I'm trying to replicate Silver & Dunlap (1987). I'm just comparing averaging correlations or averaging z transform correlations and back transforming. I seem to not be replicating the asymmetry in the bias they find (back transformed zs are not closer to the population value for me than rs). Any thoughts? Is it possible that 1987 computing power just didn't explore the space enough?

# Fisher's r2z
fr2z <- atanh
# and back
fz2r <- tanh

# a function that generates a matrix of two correlated variables
rcor <- function(n, m1, m2, var1, var2, corr12){
    require(MASS)
    Sigma <- c(var1, sqrt(var1*var2)*corr12, sqrt(var1*var2)*corr12, var2)
    Sigma <- matrix(Sigma, 2, 2)
    return( mvrnorm(n, c(m1,m2), Sigma, empirical=FALSE) )
    }

With these function it's easy to look at a bunch of correlations (basically replicate silver and dunlap 1987) and see the difference between averaging correlations and averaging z-scores and back transforming. Here's just one.

r <- 0.9
Y <- replicate(20000, rcor(10, 0, 0, 1, 1, r))
rs <- apply(Y, 3, function(x) cor(x[,1], x[,2]))
mean(rs) - r
zs <- fr2z(rs)
fz2r( mean(zs) ) - r

Just looking at the sample size of 10 and correlations of 0.1, 0.5, and 0.9 these are the results.

     rho  r bias   z bias
     0.1  -0.006   0.006
     0.5  -0.024   0.021
     0.9  -0.011   0.011

And these are derived from Table 1 of Silver & Dunlap.

     rho  r bias   z bias
     0.1  -0.007   0.003
     0.5  -0.025   0.001
     0.9  -0.011  -0.007

These are quite different results. From my test I'm seeing that it's just a matter of direction of bias, not magnitude. But, in the published paper they're finding much less magnitude with z. I couldn't find a published non-replication.

Improve this question
$\endgroup$
11
  • $\begingroup$ I'm stuck at your first two lines. They do not appear to be correct R syntax. They also appear to assume that atanh is its own inverse, but it's not: tanh is the inverse of atanh. $\endgroup$
    – whuber ♦
    Feb 2, 2012 at 1:05
  • $\begingroup$ They're just typos in the question... fixed. $\endgroup$
    – John
    Feb 2, 2012 at 1:12
  • 1
    $\begingroup$ To me, just by eye, the r bias for rho of 0.5 in the Silver & Dunlap table looks like the outlier to me. I certainly can't vouch for the quality of the journal, which appears quite new and a bit rough around the edges, but I did find this recent paper with a Google search. See, in particular, their Table 3 which, again, by eye, appears to corroborate your results. $\endgroup$
    – cardinal
    Feb 2, 2012 at 2:26
  • 1
    $\begingroup$ @whuber: Quite true. However, the UMVUE of $\rho$ in the bivariate normal case---as you may very well know---is (fairly) well-known to be $$ r \frac{\Gamma((n-2)/2)}{\Gamma(1/2)\Gamma((n-3)/2)}\int_0^1 \frac{u^{-1/2} (1-u)^{(n-5)/2}}{\sqrt{1-u(1-r^2)}} \,\mathrm{d}u \>.$$ Here $r$ is the MLE. Sometimes this estimator appears under the notation $G(r)$. $\endgroup$
    – cardinal
    Feb 2, 2012 at 2:59
  • 1
    $\begingroup$ @whuber: You raise good points. I didn't have ready access to the S&D paper either, so my remarks have been reduced to conjecture. If we ever happen to meet in person, I'll swap a story or two with you over a beer on the frustrations of dealing with those who insist on averaging correlations. I agree wholeheartedly with your comments on the matter. That said, it may make sense in some settings that I'm generally less familiar with. :) $\endgroup$
    – cardinal
    Feb 2, 2012 at 3:25

1 Answer 1

Reset to default
9
$\begingroup$

To me, the r bias entry for rho of 0.5 in the Silver & Dunlap table looks the most suspiciously different to me. However, that said, it does match your estimated value quite closely.

Unfortunately, I don't have access to the Silver & Dunlap paper at the moment, but a Google search did turn up a recent paper that performs a similar study to the one you've done. It is

R. L. Gorsuch and C. S. Lehmann (2010), Correlation coefficients: Mean bias and confidence interval distortions, Journal of Methods and Measurement in the Social Sciences, vol. 1, no. 2, 52–65.

See, in particular, their Table 3 which, at least by eye, appears to corroborate your results.

I certainly can't vouch for the quality of the journal (or the whole paper), which looks quite new and a bit rough around the edges, in my estimation. Caveat lector.

For an in-depth, more theoretical, treatment of inference on correlation (simple, partial, and multiple) primarily in a multivariate normal framework, a good reference is

F. A. Graybill, Theory and application of the linear model, Duxbury Press, 1976, Chapter 11.

It does not concern itself much with small-sample performance or applied aspects, though.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.