14
$\begingroup$

I'm curious to know if anyone has a specific reference (text or journal article) to support the common practice in the medical literature of performing sample size calculation using methods that are parametric (i.e. assuming a normal distribution and a certain variance of measurements) when the analysis of the primary trial outcome will be done using non-parametric methods.

An example: primary outcome is time to vomiting after giving a certain drug, which is known to have a mean value of 20 minutes (SD 6 minutes), but has a noticeably right-skewed distribution. The sample size calculation is done with the assumptions listed above, using the formula

$n(\text{per-group})=f(\alpha,\beta) \times (2\sigma^2 /(\mu_1 - \mu_2)^2 )$,

where $f(\alpha, \beta)$ changes based on the desired $\alpha$ and $\beta$ errors.

However, because of the skewness of the distribution, the analysis of the primary outcome will be based on ranks (non-parametric method such as the Mann Whitney U test).

Is this schema supportable by authors in the statistical literature, or should non-parametric sample size estimates be performed (and how would these be done)?

My thoughts are that, for ease of calculation, it is acceptable to do the above practice. After all, sample size estimates are just that - estimates that make several assumptions already - all of which are likely slightly (or very!) imprecise. However, I am curious to know what others think, and specifically to know if there are any references to support this line of reasoning.

Many thanks for any assistance.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
7
$\begingroup$

Some people seem to use a concept of Pitman Asymptotic Relative Efficiency (ARE) to inflate the sample size obtained by using a sample size formula for a parametric test. Ironically, in order to compute it, one has to assume a distribution again... see e.g. Sample size for the Mann-Whitney U test There are some links in the end of the article that provide pointers for further reading.

Improve this answer
$\endgroup$
6
$\begingroup$

It sounds dodgy to me. Nonparametric methods almost always involve more degrees of freedom than parametric methods and so need more data. In your particular example, the Mann-Whitney test has lower power than the t-test and so more data are required for the same specified power and size.

A simple way to do sample size calculation for any method (non-parametric or otherwise) is to use a bootstrap approach.

Improve this answer
$\endgroup$
4
  • 5
    $\begingroup$ I agree with you, although most of the sample size calculation that are done when devising RCTs are based on parametric models. I like the bootstrap approach, but it appears that very few studies rely on it. I just found those papers that might be interesting: bit.ly/djzzeS, bit.ly/atCWz3, and this one goes in the opposite direction bit.ly/cwjTHe for health measurement scales. $\endgroup$
    – chl
    Commented Sep 28, 2010 at 11:22
  • 6
    $\begingroup$ I agree about the bootstrap approach. But power is not a function of degrees of freedom. In many cases, including this one, the Mann-Whitney test often has greater power than the t-test. See tbf.coe.wayne.edu/jmasm/sawilowsky_misconceptions.pdf . In general, the power of a parametric test is good when the parametric assumptions are true but can be lower--sometimes drastically so--when those assumptions are violated, whereas good nonparametric tests maintain their power. $\endgroup$
    – whuber ♦
    Commented Nov 23, 2010 at 19:53
  • $\begingroup$ @RobHyndman - sorry to dig up an old thread from 6 years ago, but I'm wondering if you can provide a reference for your last sentence. How can I use a bootstrap approach to get a sample size calculation? I'm assuming here that I haven't gathered the data yet (because I'm trying to figure out how much to gather), but I know the power I want, the significance level, and the effect size I want to detect. Thanks! $\endgroup$
    – David White
    Commented Jul 12, 2016 at 2:06
  • $\begingroup$ Okay, I guess this can only work if you have a preliminary study to resample from. For a first time study with no prior knowledge it seems best to compute effect size from the normal distribution (or from a different distribution if theory suggests the data should be distributed that way) and add a bit to account for possible non-normality. Once you have one study you can use boostrapping to compute sample sizes to detect various effect sizes in subsequent studies. You could even fit a curve of effect size vs. n based on bootstrapping several values of n. $\endgroup$
    – David White
    Commented Jul 12, 2016 at 2:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.