1
$\begingroup$

I am trying to perform a power analysis to determine what sample size to use for an experiment after performing a pilot study. However, the sample sizes are showing values that are way too high and not feasible. Is there any other type of power analysis that could be done to justifiably lower the sample size needed for my study? I used the WMWssp function in r which calculates sample size for wilcoxin rank sum tests since I am using count data (https://rdrr.io/cran/WMWssp/man/WMWssp.html).

My code in R is

WMWssp(c(69, 4, 62, 49, 0), 
       c(131, 41, 16, 0, 78, 38, 14, 157, 16), 
       alpha = 0.05, power = 0.8, t = 'min', 
       simulation = FALSE, nsim = 10^4). 

It results in a needed sample size of 603 (297 in 1 group and 303 in the other) which is not possible to do because of financial and time limitations.

$\endgroup$
3
  • 4
    $\begingroup$ The answer is that this test requires 600 observations to achieve the power you desire at the alpha level you desire for the effect size you want to detect. Your alternatives are to try another type of test that might have higher power, lower your standards (accept higher type I error, higher type II error, or look for a larger effect size), or go with the sample size that is feasible and report the power you’d be able to get out of such a study. (For option 3, I suggest figuring out what sample sizes are feasible and calculating if the power is sufficient before you collect data.) $\endgroup$
    – Dave
    Commented Feb 16, 2020 at 20:53
  • $\begingroup$ Thanks Dave. What would be the minimum power that you would consider to be sufficient for a study in general? $\endgroup$
    – Ryan
    Commented Feb 16, 2020 at 21:58
  • $\begingroup$ I think 80% is pretty standard and you might have trouble getting funding if you cannot conduct an experiment with 80% power to detect the effect size of interest. $\endgroup$
    – Dave
    Commented Feb 16, 2020 at 22:16

2 Answers 2

2
$\begingroup$

The simple answer is "no" and that this is one reason you do a pilot study and a power analysis.

You have discovered that you do not have the resources to do what you want to do. So, rather than waste time and money doing something you can't do, you can stop until you get more funding or you can choose some other thing to study.

The above assumes that you did the pilot study correctly, that Wilcxon is the right test and that your R code is correct. A couple thoughts on that:

You have

c(69, 4, 62, 49, 0), c(131, 41, 16, 0, 78, 38, 14, 157, 16)

I assume that these are two vectors of responses from your pilot. There is huge variation within groups. Maybe you don't want to compare these numbers but their logs or something else. It's impossible for us to tell without context (log(0) would be a problem of course, but there might be a work-around).

You have

simulation = FALSE, nsim = 10^4

I don't know this R function, but this sounds like you turned off simulation and then requested an option that applies only to simulation. That sounds like a mistake.

$\endgroup$
0
$\begingroup$

Using pwr package in R and calculating sample size for t-tests, as described here, I get following result:

library(pwr)
pwr.t.test(n = , d = 0.389, sig.level = 0.05 , power = 0.8, type = "two.sample") 

Output:

 Two-sample t test power calculation 

          n = 104.707      <<<<<<<<< NOTE
          d = 0.389
  sig.level = 0.05
      power = 0.8
alternative = two.sided

NOTE: n is number in *each* group

Here, sample size of 105 is much smaller than your results.

Effect size used above was calculated as follows:

d (effect size) = (diff between means)/pooledSD
(diff between means) = 54.6 - 36.8 = 17.8
pooledSD = sqrt((sd1^2 + sd2^2)/2) : 45.7
d = 17.8/45.7 = 0.389
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.