Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The Wikipedia article Spearman's rank correlation coefficient contains an example for calculating ρ. At the end of the sections is the statement "...with a P-value = 0.6864058 (using the t distribution)." The author does not describe how the P-value was calculated from the data in the example.

How was the P-value derived for the article's specific example?

Note: The included links to Wikipedia's P-value and T-distribution entries are too generic to provide a clear answer.

share|improve this question
up vote 5 down vote accepted

In the next section in the article, "determining significance", there is the equation

$t=r\sqrt{\frac{n-2}{1-r^2}}$

If you plug in their estimate of r into that equation you get a t statistic of -0.505, which you can compare to a standard t distribution via a table or a computer. For example (EDITED- thanks to @whuber for correcting my earlier version):

> spearmentt <- function(r,n){r*sqrt((n-2)/(1-r^2))}
> test <- spearmentt(-0.17575757575,10)
> test
[1] -0.5049782
> 1- pt(test,8)
[1] 0.6864058

It's worth noting that this is a one sided test for whether r is significantly larger than zero. Probably more appropriate test would be

> pt(test,8)
[1] 0.3135942

which is a one-sided t-test for whether r is significantly less than zero ie is their evidence of a negative correlation between TV watching and IQ.

share|improve this answer
    
You should use n=10, not n=8, in invoking your "spearment" function. Indeed, r <- 1 - 6 * 194/(10*(10^2-1)); n <- 10; t <- r * sqrt((n-2)/((1-r)*(1+r))); 1 - pt(t, n-2) returns 0.6864058 as given by Wikipedia. – whuber Feb 14 '12 at 19:26
    
Thanks, I've corrected it in the main answer – Peter Ellis Feb 14 '12 at 19:52
1  
But that's the wrong t-test isn't it? They're using a one-sided t test that will return a low p value only for a high value of r. So any negative value of r (ie negative correlation of TV watching and IQ, which may well be the question) will return a value of p >0.5. – Peter Ellis Feb 14 '12 at 20:07
    
@PeterEllis Thanks. That mostly explains how they got their P-value. Can you clarify the function "pt" you used in your code? – Russell Thackston Feb 14 '12 at 20:47
1  
@RussellThackston - it's the function in R that returns the cumulative probability of getting a number in a t distribution, with given degrees of freedom, as large as the first argument. An equivalent of my one sided t-test (testing for a value of r that is significantly smaller than zero) in Excel would be '=T.DIST(-0.5049782,8,TRUE)'. – Peter Ellis Feb 14 '12 at 21:41

In R I would suggest using the functions rcorr from the Hmisc package and the custom function flattenCorrMatrix (http://www.sthda.com/english/wiki/print.php?id=78) which you can find online. rcorr calculates the correlation matrix and the flatten function will give you easy access to correlation and p values.

Something along these lines is nice for multiple variables that you want to test for correlations:

#extract columns you want to test
y<-x[,c(6:7,9:14,16,17,20,21)]

#calculates spearman correlations of all possible variable pairings
z<-rcorr(as.matrix(y),type=c("spearman"))

#flattens the matrix and gives you a nice table of all results
z1<-flattenCorrMatrix(zf$r, z$P)

# write your results in csv file
write.csv(z1, file = "z1results.csv")

Cheers

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.