# Sample size calculation for RM ANOVA; the difference between WebPower package in R and G*Power 3.1.9.7

I've run two sample size calculations with the WebPower package in R and G*Power simultaneously. The conditions I've done are;

# Case 1

Cohen's f = 0.2756987
Number of groups = 1
Number of repeated measures = 4
Non-sphericity correction = 0.672
alpha = 0.05
power = 0.80
one group, within-effect (for WebPower, within factors for G*Power)

The result is as follows;

> WebPower::wp.rmanova(ng = 1,
nm = 4,
f = 0.2756987,
nscor = 0.672,
alpha = 0.05, power = 0.80,
type = 1) -> n
> n
Repeated-measures ANOVA analysis

n         f ng nm nscor alpha power
190.5745 0.2756987  1  4 0.672  0.05   0.8

NOTE: Power analysis for within-effect test
URL: http://psychstat.org/rmanova
> n$$n [1] 190.5745 > n$$n/3+1
[1] 64.52483
> ceiling(n\$n/3+1)
[1] 65


And GPower (performed under "options -> as in SPSS," you can find the option button at the lower part of the GPower window.

F tests - ANOVA: Repeated measures, within factors
Analysis:   A priori: Compute required sample size
Input:  Effect size f(U)    =   0.2756987
α err prob  =   0.05
Power (1-β err prob)    =   0.8
Number of groups    =   1
Number of measurements  =   4
Nonsphericity correction ε  =   0.672
Output: Noncentrality parameter λ   =   9.9603207
Critical F  =   3.0568300
Numerator df    =   2.0160000
Denominator df  =   131.04
Total sample size   =   66
Actual power    =   0.8032615


What I wonder is why I should divide the calculated number in WebPower with the df of repeated measures number and add the group number (not df of group number) to get the final sample size? I referred to this method of dividing and adding some numbers from the YouTube video "Power Analysis in G*Power & RStudio (RM ANOVA)" by Lacey Maths & Stats Consultancy (https://www.youtube.com/watch?v=Ya3KdVFhcdY&t=1355s). You can recognize his explanation without detailed comments from the video 14:25.

In addition, If I run the G*Power option of "as in Cohen(1988) - recommended", then the output is;

F tests - ANOVA: Repeated measures, within factors
Analysis:   A priori: Compute required sample size
Input:  Effect size f(V)    =   0.2756987
α err prob  =   0.05
Power (1-β err prob)    =   0.8
Number of groups    =   1
Number of measurements  =   4
Nonsphericity correction ε  =   0.672
Output: Noncentrality parameter λ   =   9.7560064
Critical F  =   3.0108542
Numerator df    =   2.0160000
Denominator df  =   383.04
Total sample size   =   191
Actual power    =   0.8009450


It is same with WebPower before additional calculations. Then, Which number (191 vs 66) is a right one for me:

# Case 2

Cohen's f = 0.001630609 (I know this is a silly number, but it makes the difference between the two methods)
Number of groups = 3
Number of repeated measures = 4
Non-sphericity correction = 0.672
alpha = 0.05
power = 0.80
Three groups, interaction effect (for WebPower, within-between interaction for G*Power)

The result is as follows;

> WebPower::wp.rmanova(ng = 3,
nm = 4,
f = 0.001630609,
nscor = 0.672,
alpha = 0.05, power = 0.80,
type = 2) -> two.rmanova.n
> two.rmanova.n
Repeated-measures ANOVA analysis

n           f ng nm nscor alpha power
6696827 0.001630609  3  4 0.672  0.05   0.8

NOTE: Power analysis for interaction-effect test
URL: http://psychstat.org/rmanova
> two.rmanova.n$$n [1] 6696827 > two.rmanova.n$$n/3+3
[1] 2232279


And G*Power (as in SPSS)

F tests - ANOVA: Repeated measures, within-between interaction
Analysis:   A priori: Compute required sample size
Input:  Effect size f(U)    =   0.001630609
α err prob  =   0.05
Power (1-β err prob)    =   0.8
Number of groups    =   6
Number of measurements  =   4
Nonsphericity correction ε  =   0.672
Output: Noncentrality parameter λ   =   16.2870663
Critical F  =   1.8271026
Numerator df    =   10.0800000
Denominator df  =   6125523
Total sample size   =   3038460
Actual power    =   0.8000005


G*Power as in Cohen (1988) - recommended

F tests - ANOVA: Repeated measures, within-between interaction
Analysis:   A priori: Compute required sample size
Input:  Effect size f(V)    =   0.001630609
α err prob  =   0.05
Power (1-β err prob)    =   0.8
Number of groups    =   3
Number of measurements  =   4
Nonsphericity correction ε  =   0.672
Output: Noncentrality parameter λ   =   11.9657047
Critical F  =   2.3659662
Numerator df    =   4.0320000
Denominator df  =   1.350081e+007
Total sample size   =   6696831
Actual power    =   0.8000001


As you see, the calculated number from WebPower is the same as those of GPower (Cohen); there is no need to divide df of repeated measures number and add the number of groups.
But, G
Power (as in SPSS) results never get from the additional calculation of WebPower. Should I do additional calculations of WebPower results or Not?

This query will help with many RM ANOVA sample size-related questions. I am waiting for a brilliant answer to this. Thank you in advance for answering my question.

• FYI, I used WebPower version 0.9.3 and G*Power version 3.1.9.7 in Windows 11. Nov 7, 2023 at 1:55
• According to the WebPower Manual (webpower.psychstat.org/wiki/manual/power_of_rmanova), dividing with the number of groups seems right, not with the df and just adding the number of groups.  Sample Size The power calculation assumes the equal sample size for all groups. The total sample size is the product of the number of groups and the sample size for each group. For example, if 10 subjects are in each of the 3 groups, then the total sample size would be 3×10=30.  Nov 7, 2023 at 1:59
• Have you checked the 'Difference from the effect size in G*Power' section in the above-linked manual? Nov 7, 2023 at 2:07
• Yes, I've already checked that section. I've set the correlation among the repeated measures as 0.5 and modified the effect size, as the WebPower Manual says. However, the calculated sample size is quite different from G*Power's results. My tiny understanding is that WebPower modifies the inputted effect size, as they explained. Not require to put the modified effect size. Nov 7, 2023 at 3:33

I don't get the same output as you do from G*Power; the lowest critical value I can achieve for your case 1 is $$F>3.0819664$$ using the provided parameters. Still, I think a key point is that you are not showing all parameters. To summarize very briefly: the 'Options' dialog is hiding relevant information.

As explained in the WebPower manual, their effect size $$f$$ is not parametrized in the same way as in G*Power. Specifically, the G*Power effect is $$f_G=\frac{\sigma_m}{\sigma}$$ but the WebPower effect is:

$$f_W=\frac{\sigma_m}{\sigma}\cdot C = f_G\cdot C$$

For within-subject and interaction effects $$C = \sqrt{K/(1-\rho)}$$ with $$K$$ the number of repeated measures and $$\rho$$ their correlation. The latter I do not see anywhere in your G*Power input because it seems like the different 'Options' settings actually take charge of setting $$\rho$$. If you take full control of the effect size specification you'll see that by default $$\rho=0.5$$.

Using this default value, let's back-transform $$f_W$$ into $$f_G$$:

$$f_G = \frac{f_W}{C} = \frac{0.2756987}{\sqrt{4/0.5}}\approx0.09747421$$

Here are the results when I plug this $$f_G$$ into G*Power:

$$N=191$$ which matches WebPower with rounding up. Alternatively you can turn $$f_G=0.2756987$$ into $$f_W\approx 0.7797937$$ assuming $$\rho=0.5$$ which will give consistent results of $$N=26$$ in my case. Empirically it seems the 'SPSS' setting uses $$\rho=0.83$$ which will again allow you to derive the necessary $$f$$ to get matching results.

#### Conclusion

The different 'Options' assume a different value for within-subject correlation $$\rho$$, which by default and likely in the 'Cohen recommended' setting is $$0.5$$. I'm not sure which assumptions the 'SPSS' and 'Implicit rho' settings make exactly, but to be safe it might be best to always explicitly enter the value yourself. Also keep in mind that you cannot enter the same value of $$f$$ into both applications and expect the same result. As to which sample size numbers are 'correct': that will depend on the true value of $$\rho$$...

• +1. GPower's developers should probably improve their interface, it's not the first time I see questions/confusion created by these hidden options. Note that the "determine" button gives even more hidden options. On the question of what the different options do exactly, this other related discussion is certainly relevant: stats.stackexchange.com/questions/520090/… Nov 8, 2023 at 11:29