3
$\begingroup$

I am performing a prospective-design analysis of Patient Reported Outcome Measures from a registry of patients. I have used Shapiro-Wilk test throughout to test for normality.

All data is paired and non-parametric - I have therefore employed Wilcoxon Signed-Rank tests to assess statistical significance.

Calculating statistical power seems to be straightforward for paired/non-paired t-tests. However, I do not know how to calculate power for the Wilcoxon test.

The cohort sample size is n=~200, but this is reduced after records at 1, 3, and 6 months, which is why I would like to calculate statistical power to justify my conclusions.

Would anyone be able to point me in the right direction. This is my first time posting here so let me know if I need to add any more info.

thank you :)

$\endgroup$

1 Answer 1

1
$\begingroup$

This is not really an answer to your question because I'm not sure there is a useful general answer. You say, "Calculating statistical power seems to be straightforward for paired/non-paired t-tests." Let's start there.

Background on the power of t tests. For example suppose we want to find power for a two-sample pooled t test. What do you need to know in order to get the sample size $n = n_1 = n_2$ in each group in order to detect a difference $\Delta = \mu_1 - \mu_2$ (or larger) with $90\%$ power in a one-sided test with significance level $\alpha = $5%.$

In addition to $\alpha,$ $\Delta$ and the desired power, you need to know the common population standard deviation $\sigma_1 = \sigma_2 = \sigma.$ More specifically, you need to know $\Delta/\sigma.$ Suppose, you want $\Delta/\sigma = 0.5.$ Then you are ready to use the 'power and sample size' procedure in some statistical software to find the required $n = 70.$ Results from Minitab are shown below, based on the noncentral t distribution. The power curve shows the power for other values of $\Delta$ when other quantities are the same.

Power and Sample Size 

2-Sample t Test

Testing mean 1 = mean 2 (versus >)
Calculating power for mean 1 = mean 2 + difference
α = 0.05  Assumed standard deviation = 2

            Sample  Target
Difference    Size   Power  Actual Power
         1      70     0.9      0.902966

The sample size is for each group.

enter image description here

A similar computation for the Welch one-sided, two-sample t test is a little messier because the Welch test does not require $\sigma_1 = \sigma_2$ and compensates for unequal group variances by decreasing the degrees of freedom based on the sample standard deviations and sample sizes. For a specific case, I can use simulation to check whether $n = 70$ subjects in each group is still sufficient for $90\%$ power if $\sigma_1 = 1.5, \sigma_2 = 2.5.$ The answer is Almost, the power drops to a little over $89%.

set.seed(2010)
pv = replicate(10^5, t.test(rnorm(70, 50, 1.5), 
                        rnorm(70, 51, 2.5), alt="less")$p.val)
mean(pv <= .05)
[1] 0.88639    # Rejection probability = power

Specific information required for power of Wilcoxon RS test. Now for nonparametric two-sample Wilcoxon rank sum tests. Somehow we would need to know the effect size $\Delta$ and the variability of the data in order to get the required sample size for a test at level $\alpha.$ If you could be sufficiently specific about the two populations being compared, you could use simulation to find the power of the Wilcoxon test for a given number $n$ of subjects in each group. It is not enough just to know that the population distributions are not normal.

If you had evidence that the two populations are $\mathsf{Gamma}(\mathrm{shape}=5, \mathrm{rate}=0.13)$ and $\mathsf{Gamma}(5, 0.1),$ which are of roughly the same shape, you could use a Wilcoxon RS test to see if the populations have different medians. For $n = 70$ subjects in each group the power would be about $95\%.$

set.seed(2010)
pv = replicate(10^5, wilcox.test(rgamma(70, 5, 0.13), 
                        rgamma(70, 5, 0.1), alt="less")$p.val)
mean(pv <= .05)
[1] 0.94822

The two medians are about $\eta_1 = 35.9, \eta_2 = 46.7.$

qgamma(.5, 5, 0.13);  qgamma(.5, 5, 0.1)
[1] 35.93007
[1] 46.70909

However, if we have such specific information about the two population distributions, we might be able to find an exact test, using the mathematical forms of the density functions, which could be more powerful than the Wilcoxon SR test. Perhaps the main difficulty finding the power of a nonparametric test is that we do not usually think about the specific form of the populations being compared. The term 'nonparametric' often means 'not necessarily just for normal populations'. It does not forbid application to distributions that have density functions involving parameters.

$\endgroup$
1
  • $\begingroup$ +1 "Somehow we would need to know the effect size." EXACTLY! $\endgroup$
    – Alexis
    Commented Sep 23, 2022 at 23:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.