Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm curious to know if anyone has a specific reference (text or journal article) to support the common practice in the medical literature of performing sample size calculation using methods that are parametric (i.e. assuming a normal distribution and a certain variance of measurements) when the analysis of the primary trial outcome will be done using non-parametric methods.

An example: primary outcome is time to vomiting after giving a certain drug, which is known to have a mean value of 20 minutes (SD 6 minutes), but has a noticeably right-skewed distribution. The sample size calculation is done with the assumptions listed above, using the formula

$n(\text{per-group})=f(\alpha,\beta) \times (2\sigma^2 /(\mu_1 - \mu_2)^2 )$,

where $f(\alpha, \beta)$ changes based on the desired $\alpha$ and $\beta$ errors.

However, because of the skewness of the distribution, the analysis of the primary outcome will be based on ranks (non-parametric method such as the Mann Whitney U test).

Is this schema supportable by authors in the statistical literature, or should non-parametric sample size estimates be performed (and how would these be done)?

My thoughts are that, for ease of calculation, it is acceptable to do the above practice. After all, sample size estimates are just that - estimates that make several assumptions already - all of which are likely slightly (or very!) imprecise. However, I am curious to know what others think, and specifically to know if there are any references to support this line of reasoning.

Many thanks for any assistance.

share|improve this question
add comment

2 Answers

It sounds dodgy to me. Nonparametric methods almost always involve more degrees of freedom than parametric methods and so need more data. In your particular example, the Mann-Whitney test has lower power than the t-test and so more data are required for the same specified power and size.

A simple way to do sample size calculation for any method (non-parametric or otherwise) is to use a bootstrap approach.

share|improve this answer
3  
I agree with you, although most of the sample size calculation that are done when devising RCTs are based on parametric models. I like the bootstrap approach, but it appears that very few studies rely on it. I just found those papers that might be interesting: bit.ly/djzzeS, bit.ly/atCWz3, and this one goes in the opposite direction bit.ly/cwjTHe for health measurement scales. –  chl Sep 28 '10 at 11:22
3  
I agree about the bootstrap approach. But power is not a function of degrees of freedom. In many cases, including this one, the Mann-Whitney test often has greater power than the t-test. See tbf.coe.wayne.edu/jmasm/sawilowsky_misconceptions.pdf . In general, the power of a parametric test is good when the parametric assumptions are true but can be lower--sometimes drastically so--when those assumptions are violated, whereas good nonparametric tests maintain their power. –  whuber Nov 23 '10 at 19:53
add comment

Some people seem to use a concept of Pitman Asymptotic Relative Efficiency (ARE) to inflate the sample size obtained by using a sample size formula for a parametric test. Ironically, in order to compute it, one has to assume a distribution again... see e.g. Sample size for the Mann-Whitney U test There are some links in the end of the article that provide pointers for further reading.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.