6
$\begingroup$

I am wondering how to derive the formula for the standard error of Pearson's correlation coefficient which is given in Zar for example as

$$ \newcommand{\cov}{{\rm Cov}} \newcommand{\var}{{\rm Var}} \newcommand{\sd}{{\rm SD}} SE_r =\sqrt{\frac{1-r^2}{n-2}}$$

I tried to get it from estimating the variance of r when

$$r =\frac{\cov(x,y)}{\sd(x)\sd(y)}$$

and $V(X) = E(X^2) - E(X)^2$ so we get $Var(r) = E\bigg(\frac{\cov(x,y)^2}{\var(x)\var(y)}\bigg) - r^2$. But from here I don't know how to continue since $E\bigg(\frac{\cov(x,y)^2}{\var(x)\var(y)}\bigg)$ would have to be $\frac{1-(n-3)r^2}{n-2}$ to get finally to

$$\var(r) =\frac{1-r^2}{n-2}$$

Any suggestions or references where I could look this up?

$\endgroup$
6
  • $\begingroup$ Maybe that helps en.youscribe.com/catalogue/tous/knowledge/formal-sciences/… $\endgroup$ – Robert Jul 30 '16 at 1:41
  • $\begingroup$ Thanks for the link but it just lists lots of formulas for SE of particular estimates but doesn't give any information of how these formulas are obtained. The fact that this cannot be found by using google somehow tells me that people just use it without knowing where it comes from which i find a bit scary... $\endgroup$ – R. Carlos Jul 30 '16 at 12:57
  • 1
    $\begingroup$ That's a really good question. Contrary to many textbooks,, that is not the standard error of r. For starters, the standard error of a statistic cannot be defined in terms of a statistic. It should be defined in terms of parameters. I believe you use rho in the formula and divide by sqrt(N-1) rather than 2 but I don't have a reference handy. $\endgroup$ – David Lane Mar 16 '17 at 2:11
  • 2
    $\begingroup$ One more reference: tandfonline.com/doi/abs/10.1080/… $\endgroup$ – David Lane Mar 16 '17 at 2:53
  • 4
    $\begingroup$ From all the references above and below, it seems that the formula you want to recover is wrong, anyway... $\endgroup$ – Pascal Mar 13 '18 at 12:03
5
$\begingroup$

After looking for a long time for an answer to this same question, I found a couple interesting links https://www.jstor.org/stable/2277400?seq=1#page_scan_tab_contents

where we can only see the first page but that's where the derivation is. The "standard deviation by dr Sheppard" is given by something called the Asymptotic distribution of moments, of which you can see a bit here

https://books.google.com/books?id=Uc9C90KKW_UC&pg=PA126&lpg=PA126&dq=Mst+pearson+Sheppard&source=bl&ots=Kvw0xTLzps&sig=pyHVB_ybjsnb_0QOBDHST6SRi-M&hl=en&sa=X&ved=0ahUKEwimjvjQ8NnSAhWEppQKHRqbC1sQ6AEIIjAD#v=onepage&q=Mst%20pearson%20Sheppard&f=false

The reason for the "n-2" instead of "n" in the root, is that your formula assumes a t-distribution with n-2 degrees of freedom, while the one in the links assumes a normal distribution.

$\endgroup$
1
  • 1
    $\begingroup$ It looks like the equal sign in the formula should be written as an approximately sign then cause it reflect asymptotic behaviour rather than a closed form solution. Further, the SE of the correlation coefficient is also then heavily dependent on the sample size as far as I read it from Pearson original papers. Conclusion is that the formula for the SE of r cannot be simply obtained as it is the case for the SE of the mean or the variance. $\endgroup$ – R. Carlos Apr 24 '17 at 2:22
1
$\begingroup$

I do not have the answer, but for me there is an error in the formula of the question.

It is: $$SE_r =\frac{1-r^2}{\sqrt{n-2}}$$ and $$\newcommand{\Var}{\operatorname{Var}} \Var(r)=\frac{(1-r^2)^2}{n-2}$$

I will try to check this by simulation:

library(MASS)

N = 100000
r = 0.8
n = 100

Sigma = matrix(c(1, r, r, 1), nrow=2)
r_obs =  replicate(N, cor(mvrnorm(n, c(0,0), Sigma))[2,1])

> mean(r_obs)
[1] 0.7984783
> sd(r_obs)
[1] 0.03690896

So the standard error for r=0.8 with n=100 is approximately 0.037.

If I use the formula of the question I get:

> sqrt((1-r^2)/(n-2))
[1] 0.06060915

And with the formula I gave:

> (1-r^2)/sqrt((n-2))
[1] 0.03636549

The second formula seems to be much closer to the true value than the first.

Edit:

To try to explain why there are two different formulas for the standard error which are circulating, I found that it depends on how you compute it.

In my first simulation, I used Pearson formula to compute the correlation, but one can also use the least square regression coefficient. I can confirm that the latter method has the standard error proposed in the question:

r_obs <- replicate(N, {
    M<-mvrnorm(n, c(0,0), Sigma)
    c(pearson=cor(M, method="pearson")[2,1],
      regression=lm(M[,2]~M[,1])$coef[[2]])
    })

> apply(r_obs, 1, mean)
   pearson regression 
 0.7981580  0.7998433 
> apply(r_obs, 1, sd)
   pearson regression 
0.03707184 0.06094964 

These are two estimators of the correlation which do not have the same variance.

Edit 2:

That try to reconcile the two formulas did not work, because I forgot to normalize the regression coefficient. The formula to compute the correlation from the regression coefficient is: $$r=\frac{SD(x)}{SD(y)}b$$ with $$E(y|x)=a+b\cdot x$$

By redoing the correct computation, I obtain in fact the same result:

r_obs <- replicate(N, {
    M<-mvrnorm(n, c(0,0), Sigma)
    c(pearson=cor(M, method="pearson")[2,1],
      regression=lm(M[,2]~M[,1])$coef[[2]]*sd(M[,1])/sd(M[,2]))
    })
> apply(r_obs, 1, sd)
   pearson regression 
0.03676248 0.03676248 
> apply(r_obs, 1, mean)
   pearson regression 
 0.7992615  0.7992615 

Which is reassuring in some way. So I maintain that the standard error formula in the question is incorrect, but maybe I explained where does the error come from.

$\endgroup$
3
  • 1
    $\begingroup$ There is no error in the original formula: see en.wikipedia.org/wiki/Pearson_correlation_coefficient. You will discover your formula doesn't seem to work, either, if you set n <- 3 instead of 100 and r <- 0 instead of 0.8. There are several problems, one of which is that the formula of the question is a reasonable standard error only when $r$ is close to zero. Another is that the sampling distribution can be highly skewed, rendering any SE less than useful: that's why better analyses re-express $r$ with the Fisher transformation. $\endgroup$ – whuber Nov 6 '18 at 16:02
  • 2
    $\begingroup$ I think that n=3 is a too special case to judge. I don't know, maybe n-2 is not the good divisor, maybe it is n-3. Beyond that, the two formulae give the same result for r=0. $\endgroup$ – Jean Paul Nov 6 '18 at 17:05
  • 1
    $\begingroup$ Please explain the downvotes. In addition, I would like to know where I can find the formula of the standard error of the raw correlation coefficient (not the Fisher transform) in the Wikipedia page. Logically it should be in this section: en.wikipedia.org/wiki/Pearson_correlation_coefficient#Inference, but I am unable to see it. $\endgroup$ – Jean Paul Oct 10 '19 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.