At the moment I have $5$ paired samples for correlation. Spearman's R is $0.2$ and the $p = 0.78$.

How do I calculate the number to extra samples I would need to get a more significant p-value?

  • $\begingroup$ not homework, I'm trying to determine how many patient samples extra i will need to do for my PhD project. $\endgroup$ – Chan Jul 25 '12 at 7:28
  • $\begingroup$ @Chan, you appear to be asking about "post-hoc" power analysis - the problem is that since you only have $n=5$, your estimate of $.2$ is not a very good one. Are you suggesting doing the power analysis supposing that $.2$ is the true value of the Spearman's $\rho$? $\endgroup$ – Macro Jul 25 '12 at 11:23
  • $\begingroup$ Here's a related question (though treating Pearson's correlation coefficient): stats.stackexchange.com/q/17371/2970 $\endgroup$ – cardinal Jul 25 '12 at 14:07
  • $\begingroup$ Thank you everyone for helping. At the moment i have only n=5, but think i will do more to increase power. Spearman's rho should increase but i think the p-value would remain low as i only have a small sample size. $\endgroup$ – Chan Jul 25 '12 at 23:23

I take it that you are investigating whether the correlation between two quantities is larger than $0$ and that you wish to know how many patients you need for your study to be able to show that it really is larger. In other words, I assume that you are using a one-sided test.

First of all, even if you collect a million samples, there is no guarantee that you will get a significant result. If the correlation actually is $0$, then you likely won't get a significant result. But even if it is non-zero, there is always a possibility that you, due to randomness, won't get a significant result.

Second, how large the sample needs to be depends on how large the true correlation is.

I ran a quick computer simulation ($10,000$ repetitions) to investigate how large the sample size needs to be in order to get a high probability of a significant result. It is based on the assumption that the quantities that you measure are normally distributed. If that is not the case, then these calculations will be in error. Not necessarily a large error, but nevertheless in error.

The plots below show what the probability of getting a significant ($p<0.05$) result (called the power of the test) for different sample sizes ($n$) and different true values of the population correlation (rho=$\rho$):

Power of one-sided correlation test

If $\rho=0.2$ and $n=80$, the probability of a significant result is roughly $50~\%$. If $\rho=0.1$ and $n=80$, the probability is about $20~\%$. As you can see, it is easier to detect a large correlation than a small one.

What is typically done in these cases is to say "if $\rho=0.2$ then I want at least an $80~\%$ probability of a significant result" and to choose the smallest $n$ that satisifies that condition.

As a final remark, there are sequential sampling methods where you collect more samples until you get a significant result, but there are some caveats to them. If you're thinking of using such a sampling strategy I recommend that you consult a statistican to make sure that you use it in the right way.

  • $\begingroup$ +1 Very complete answer! I only disagree with the one-sidedness of the test but this is another discussion. $\endgroup$ – gui11aume Jul 25 '12 at 10:40
  • $\begingroup$ Cheers @gui11aume! I agree that it isn't clear whether the OP should use a one-sided test (and of course that is why I stated this assumption explicitly in my answer). $\endgroup$ – MånsT Jul 25 '12 at 10:57
  • $\begingroup$ This is a good information to present so +1 but the OP appears to be talking about Spearman's $\rho$, not the pearson product moment correlation - is that what you're using? $\endgroup$ – Macro Jul 25 '12 at 11:22
  • 1
    $\begingroup$ @Macro: I used Spearman's $\rho$ for the tests, but the "population correlation" that I speak about is the Pearson one. Not ideal, I know... $\endgroup$ – MånsT Jul 25 '12 at 12:06

It depends on the test that was performed and the assumptions you make. I will assume that you want to calculate the sample size that would give a significant p-value at 0.05 given that the value of $\rho$ is still 0.2.

You can use the approximation

$$t = \rho\sqrt{\frac{n-2}{1-\rho^2}}$$

where $t$ has an approximate Student's t distribution. Since the value of $\rho$ is fixed, you need to find the value of $n$ such that $t$ is the 97.5-th percentile of a t distribution witn $n-2$ degrees of freedom. In your case I find a cutoff for $n = 97$.

  • $\begingroup$ Do you mean that i will need to increase my sample size to 97? I only have 20 patients in the cohort. with 5 patients it already shows a positive calculation, so it shouldn't matter if R gets higher, just require the p-value to hopefully be less than 0.05. $\endgroup$ – Chan Jul 25 '12 at 7:56
  • $\begingroup$ I am not sure I understand your logic. When there is no effect, you still have a 1/2 chance that the computed correlation is positive. You cannot just decide the sample size you need to observe a significant correlation. I simply said that if the correlation were 0.2 on a bigger sample, this sample should have size 97. Anyway, if you have 20 patients in the cohort, just test them all. $\endgroup$ – gui11aume Jul 25 '12 at 8:07
  • $\begingroup$ @su11aume If the OP is doing this in two-stages he needs to adjust for multiplicity. This could be done with a two stage adaptive design with sample size readjustment at the first stage (based on conditional power or effect size estimates). This issue is pointed out in MansTs answer. $\endgroup$ – Michael R. Chernick Jul 25 '12 at 11:20
  • $\begingroup$ @MichaelChernick yes, that's right. I neglected this but should not have. $\endgroup$ – gui11aume Jul 25 '12 at 11:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.