# What is the correct way of generating a p-value for a correlation with resampling?

I have a vector of gene expression values, across 20 patients. Each patient also has a glucose measure (a continuous numeric vector). I want to find how significant the real correlation value is compared to randomly distributed correlation values.

1. Real Pearson calculated by:

real<-cor(gene1, glucose)

2. I generate a distribution of 10K random correlations using sample with replacement

random_glucose <- replicate(10000,
cor(sample(gene1,replace=T),sample(glucose,replace=T)))


How do I get a p-value of how significant the real is compared to the random values?

I have tried the following:

1. Using pnorm with the code:

pnorm_p <- 2*pnorm(q=abs(real),mean=mean(random_glucose),sd=sd(random_glucose),lower.tail=F)

2. I am not convinced the random values are normally distributed, so am also using ecdf with the code:

ecdf_calc<-function(real, random_glucose){
funny<-ecdf(random_glucose)
value<-funny(real)
if(value<=0.5){
res<-2*value
return(res)
}else{
res<-((1 - value)*2)
return(res)
}
}


What are the right ways to do it with ecdf and pnorm?

I see different ways of doing this via Google and I don't understand the difference.

E.g., methods using Z-scores as here; http://www.cyclismo.org/tutorial/R/pValues.html#calculating-a-single-p-value-from-a-normal-distribution

And sometimes I see 1-pnorm(...) and others I see pnorm(..) alone without the 1-.

• For a one tailed test, you just count how far in it is from the end. For two tailed it's a little more complicated, but still quite simple. – Glen_b Apr 23 '14 at 15:02
• You didn't ask about it but your resampling approach is somewhat odd. If this is a bootstrap, you ought to resample actual pairs of gene-glucose values. If this is a permutation test, you ought to resample values without replacement - effectively shuffling one of the value vectors (either genes or glucose values). In your case I'd prefer a permutation approach over bootstrap (given enough replications the permutation-test p-value is exact, with bootstrapping it's hard to be sure). – Trisoloriansunscreen Apr 23 '14 at 17:23