7
$\begingroup$

The Wikipedia article Spearman's rank correlation coefficient contains an example for calculating ρ. At the end of the sections is the statement "...with a P-value = 0.6864058 (using the t distribution)." The author does not describe how the P-value was calculated from the data in the example.

How was the P-value derived for the article's specific example?

Note: The included links to Wikipedia's P-value and T-distribution entries are too generic to provide a clear answer.

$\endgroup$

3 Answers 3

12
$\begingroup$

In the next section in the article, "determining significance", there is the equation

$t=r\sqrt{\frac{n-2}{1-r^2}}$

If you plug in their estimate of r into that equation you get a t statistic of -0.505, which you can compare to a standard t distribution via a table or a computer. For example (EDITED- thanks to @whuber for correcting my earlier version):

> spearmentt <- function(r,n){r*sqrt((n-2)/(1-r^2))}
> test <- spearmentt(-0.17575757575,10)
> test
[1] -0.5049782
> 1- pt(test,8)
[1] 0.6864058

It's worth noting that this is a one sided test for whether r is significantly larger than zero. Probably more appropriate test would be

> pt(test,8)
[1] 0.3135942

which is a one-sided t-test for whether r is significantly less than zero, i.e., is there evidence of a negative correlation between TV watching and IQ.

$\endgroup$
6
  • 1
    $\begingroup$ You should use n=10, not n=8, in invoking your "spearment" function. Indeed, r <- 1 - 6 * 194/(10*(10^2-1)); n <- 10; t <- r * sqrt((n-2)/((1-r)*(1+r))); 1 - pt(t, n-2) returns 0.6864058 as given by Wikipedia. $\endgroup$
    – whuber
    Commented Feb 14, 2012 at 19:26
  • 1
    $\begingroup$ But that's the wrong t-test isn't it? They're using a one-sided t test that will return a low p value only for a high value of r. So any negative value of r (ie negative correlation of TV watching and IQ, which may well be the question) will return a value of p >0.5. $\endgroup$ Commented Feb 14, 2012 at 20:07
  • $\begingroup$ @PeterEllis Thanks. That mostly explains how they got their P-value. Can you clarify the function "pt" you used in your code? $\endgroup$ Commented Feb 14, 2012 at 20:47
  • $\begingroup$ I think you're right, Peter. Indeed, the two-sided value would be 2*pt(-abs(test),8). $\endgroup$
    – whuber
    Commented Feb 14, 2012 at 21:18
  • 2
    $\begingroup$ @RussellThackston - it's the function in R that returns the cumulative probability of getting a number in a t distribution, with given degrees of freedom, as large as the first argument. An equivalent of my one sided t-test (testing for a value of r that is significantly smaller than zero) in Excel would be '=T.DIST(-0.5049782,8,TRUE)'. $\endgroup$ Commented Feb 14, 2012 at 21:41
1
$\begingroup$

In R I would suggest using the functions rcorr from the Hmisc package and the custom function flattenCorrMatrix (http://www.sthda.com/english/wiki/print.php?id=78) which you can find online. rcorr calculates the correlation matrix and the flatten function will give you easy access to correlation and p values.

Something along these lines is nice for multiple variables that you want to test for correlations:

#extract columns you want to test
y<-x[,c(6:7,9:14,16,17,20,21)]

#calculates spearman correlations of all possible variable pairings
z<-rcorr(as.matrix(y),type=c("spearman"))

#flattens the matrix and gives you a nice table of all results
z1<-flattenCorrMatrix(zf$r, z$P)

# write your results in csv file
write.csv(z1, file = "z1results.csv")

Cheers

$\endgroup$
0
$\begingroup$

Spearman correlation coefficient is a Pearson correlation coefficient calculated on the ranks of observations. Hence, you can apply p-value formulae of Pearson correlation $r$ to Spearman $\rho$.

For instance, the estimator of the variance is $\sigma^2_\rho=\frac{(n-2)}{1-\rho^2}$, you can use it in usual way to calculate the t-stat $t=\rho/\sigma_\rho$, then lookup the p-value from a table or from the CDF of t-distribution with $n-2$ degrees of freedom.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.