3
$\begingroup$

I am trying to calculate power for 150 samples where 75 are going to be in one group and 75 in another. I tried using the pwr package in R to get the power where I used the following code:

pwr.anova.test(k=2, n =75, f=.1, sig.level=.05, power=NULL)

k for me equals 2 because the 150 samples are divided into two groups equally (75 each)

n which is the sample size is 75 for both groups

f is the effect size which I set to 0.1

significance level I set to 0.05

I get a number for power which is 0.23 which obviously is low. My question is, am I doing this correctly? I am new to power calculations and if someone could help me understand how to calculate power knowing the sample size or point me to a package that can correctly do this for my example, I would truly appreciate this.

$\endgroup$
12
  • 2
    $\begingroup$ That's a small effect size, one you are unlikely to detect at a significance level of 0.05. $\endgroup$
    – whuber
    Commented Mar 21, 2023 at 17:37
  • 1
    $\begingroup$ Do you want to detect an effect size of 0.1 or 0.3? That is, how okay are you with missing an effect size of 0.1 that truly exists? $\endgroup$
    – Dave
    Commented Mar 21, 2023 at 17:41
  • 1
    $\begingroup$ You typically (or at can) do the power calculation before you collect data so you know how much data collection to do, so any notion that you must do calculations based on the data to calculate power must be false. $\endgroup$
    – Dave
    Commented Mar 21, 2023 at 17:51
  • 1
    $\begingroup$ ANOVA with two groups is equivalent to a two-sided, equal-variance, two-sample t-test (though the effect size ls in the two functions seem to refer to different quantities). $\endgroup$
    – Dave
    Commented Mar 21, 2023 at 18:04
  • 1
    $\begingroup$ However, beware --- the definition of effect size in anova and t tests is related but different. $\endgroup$
    – Glen_b
    Commented Mar 21, 2023 at 21:59

1 Answer 1

3
$\begingroup$

I get a number for power which is 0.23 which obviously is low. My question is, am I doing this correctly?

It is low, but not incorrect. When you have an effect of 0.1 and a sample of size 75, then you don't get very much power and 0.23 is not weird.


It may help to do a manual computation.

You can compute this easily manually for the case of a z-test. (Your example is an F-test but the relationship between power and effect size is more or less the same)

Say you have a distribution $X \sim N(\mu,1)$ and you test the hypothesis that $\mu = 0$, by using a sample of size $75$. Then te standard deviation of the statistic $\bar{X}$, the average of the 75 values, is $\frac{1}{\sqrt{75}} \approx 0.115$, and the effect size of $0.1$ is only a shift by one standard deviation. In the image below you see what this means

example

the cutoff values are around $\pm 0.226$ and the power is here only $0.139$, even less than your situation (with a one sided test, like the F-test, the power for a given effect will be higher, and in the case of the z-test it will be approximately $0.218$).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.