1
$\begingroup$

I've run two sample size calculations with the WebPower package in R and G*Power simultaneously. The conditions I've done are;

Case 1

Cohen's f = 0.2756987
Number of groups = 1
Number of repeated measures = 4
Non-sphericity correction = 0.672
alpha = 0.05
power = 0.80
one group, within-effect (for WebPower, within factors for G*Power)

The result is as follows;

> WebPower::wp.rmanova(ng = 1,
                       nm = 4,
                       f = 0.2756987,
                       nscor = 0.672,
                       alpha = 0.05, power = 0.80,
                       type = 1) -> n
> n
Repeated-measures ANOVA analysis

           n         f ng nm nscor alpha power
    190.5745 0.2756987  1  4 0.672  0.05   0.8

NOTE: Power analysis for within-effect test
URL: http://psychstat.org/rmanova
> n$n
[1] 190.5745
> n$n/3+1
[1] 64.52483
> ceiling(n$n/3+1)
[1] 65

And GPower (performed under "options -> as in SPSS," you can find the option button at the lower part of the GPower window.

F tests - ANOVA: Repeated measures, within factors
Analysis:   A priori: Compute required sample size 
Input:  Effect size f(U)    =   0.2756987
    α err prob  =   0.05
    Power (1-β err prob)    =   0.8
    Number of groups    =   1
    Number of measurements  =   4
    Nonsphericity correction ε  =   0.672
Output: Noncentrality parameter λ   =   9.9603207
    Critical F  =   3.0568300
    Numerator df    =   2.0160000
    Denominator df  =   131.04
    Total sample size   =   66
    Actual power    =   0.8032615

What I wonder is why I should divide the calculated number in WebPower with the df of repeated measures number and add the group number (not df of group number) to get the final sample size? I referred to this method of dividing and adding some numbers from the YouTube video "Power Analysis in G*Power & RStudio (RM ANOVA)" by Lacey Maths & Stats Consultancy (https://www.youtube.com/watch?v=Ya3KdVFhcdY&t=1355s). You can recognize his explanation without detailed comments from the video 14:25.

In addition, If I run the G*Power option of "as in Cohen(1988) - recommended", then the output is;

F tests - ANOVA: Repeated measures, within factors
Analysis:   A priori: Compute required sample size 
Input:  Effect size f(V)    =   0.2756987
    α err prob  =   0.05
    Power (1-β err prob)    =   0.8
    Number of groups    =   1
    Number of measurements  =   4
    Nonsphericity correction ε  =   0.672
Output: Noncentrality parameter λ   =   9.7560064
    Critical F  =   3.0108542
    Numerator df    =   2.0160000
    Denominator df  =   383.04
    Total sample size   =   191
    Actual power    =   0.8009450

It is same with WebPower before additional calculations. Then, Which number (191 vs 66) is a right one for me:

Case 2

Cohen's f = 0.001630609 (I know this is a silly number, but it makes the difference between the two methods)
Number of groups = 3
Number of repeated measures = 4
Non-sphericity correction = 0.672
alpha = 0.05
power = 0.80
Three groups, interaction effect (for WebPower, within-between interaction for G*Power)

The result is as follows;

> WebPower::wp.rmanova(ng = 3,
                       nm = 4,
                       f = 0.001630609,
                       nscor = 0.672,
                       alpha = 0.05, power = 0.80,
                       type = 2) -> two.rmanova.n
> two.rmanova.n
Repeated-measures ANOVA analysis

          n           f ng nm nscor alpha power
    6696827 0.001630609  3  4 0.672  0.05   0.8

NOTE: Power analysis for interaction-effect test
URL: http://psychstat.org/rmanova
> two.rmanova.n$n
[1] 6696827
> two.rmanova.n$n/3+3
[1] 2232279

And G*Power (as in SPSS)

F tests - ANOVA: Repeated measures, within-between interaction
Analysis:   A priori: Compute required sample size 
Input:  Effect size f(U)    =   0.001630609
    α err prob  =   0.05
    Power (1-β err prob)    =   0.8
    Number of groups    =   6
    Number of measurements  =   4
    Nonsphericity correction ε  =   0.672
Output: Noncentrality parameter λ   =   16.2870663
    Critical F  =   1.8271026
    Numerator df    =   10.0800000
    Denominator df  =   6125523
    Total sample size   =   3038460
    Actual power    =   0.8000005

G*Power as in Cohen (1988) - recommended

F tests - ANOVA: Repeated measures, within-between interaction
Analysis:   A priori: Compute required sample size 
Input:  Effect size f(V)    =   0.001630609
    α err prob  =   0.05
    Power (1-β err prob)    =   0.8
    Number of groups    =   3
    Number of measurements  =   4
    Nonsphericity correction ε  =   0.672
Output: Noncentrality parameter λ   =   11.9657047
    Critical F  =   2.3659662
    Numerator df    =   4.0320000
    Denominator df  =   1.350081e+007
    Total sample size   =   6696831
    Actual power    =   0.8000001

As you see, the calculated number from WebPower is the same as those of GPower (Cohen); there is no need to divide df of repeated measures number and add the number of groups.
But, G
Power (as in SPSS) results never get from the additional calculation of WebPower. Should I do additional calculations of WebPower results or Not?

This query will help with many RM ANOVA sample size-related questions. I am waiting for a brilliant answer to this. Thank you in advance for answering my question.

$\endgroup$
4
  • $\begingroup$ FYI, I used WebPower version 0.9.3 and G*Power version 3.1.9.7 in Windows 11. $\endgroup$
    – DK LEE
    Nov 7, 2023 at 1:55
  • $\begingroup$ According to the WebPower Manual (webpower.psychstat.org/wiki/manual/power_of_rmanova), dividing with the number of groups seems right, not with the df and just adding the number of groups. ` Sample Size The power calculation assumes the equal sample size for all groups. The total sample size is the product of the number of groups and the sample size for each group. For example, if 10 subjects are in each of the 3 groups, then the total sample size would be 3×10=30. ` $\endgroup$
    – DK LEE
    Nov 7, 2023 at 1:59
  • $\begingroup$ Have you checked the 'Difference from the effect size in G*Power' section in the above-linked manual? $\endgroup$
    – PBulls
    Nov 7, 2023 at 2:07
  • $\begingroup$ Yes, I've already checked that section. I've set the correlation among the repeated measures as 0.5 and modified the effect size, as the WebPower Manual says. However, the calculated sample size is quite different from G*Power's results. My tiny understanding is that WebPower modifies the inputted effect size, as they explained. Not require to put the modified effect size. $\endgroup$
    – DK LEE
    Nov 7, 2023 at 3:33

1 Answer 1

3
$\begingroup$

I don't get the same output as you do from G*Power; the lowest critical value I can achieve for your case 1 is $F>3.0819664$ using the provided parameters. Still, I think a key point is that you are not showing all parameters. To summarize very briefly: the 'Options' dialog is hiding relevant information.

As explained in the WebPower manual, their effect size $f$ is not parametrized in the same way as in G*Power. Specifically, the G*Power effect is $f_G=\frac{\sigma_m}{\sigma}$ but the WebPower effect is:

$$ f_W=\frac{\sigma_m}{\sigma}\cdot C = f_G\cdot C $$

For within-subject and interaction effects $C = \sqrt{K/(1-\rho)}$ with $K$ the number of repeated measures and $\rho$ their correlation. The latter I do not see anywhere in your G*Power input because it seems like the different 'Options' settings actually take charge of setting $\rho$. If you take full control of the effect size specification you'll see that by default $\rho=0.5$.

Using this default value, let's back-transform $f_W$ into $f_G$:

$$ f_G = \frac{f_W}{C} = \frac{0.2756987}{\sqrt{4/0.5}}\approx0.09747421 $$

Here are the results when I plug this $f_G$ into G*Power: G-Power with n=191

$N=191$ which matches WebPower with rounding up. Alternatively you can turn $f_G=0.2756987$ into $f_W\approx 0.7797937$ assuming $\rho=0.5$ which will give consistent results of $N=26$ in my case. Empirically it seems the 'SPSS' setting uses $\rho=0.83$ which will again allow you to derive the necessary $f$ to get matching results.

Conclusion

The different 'Options' assume a different value for within-subject correlation $\rho$, which by default and likely in the 'Cohen recommended' setting is $0.5$. I'm not sure which assumptions the 'SPSS' and 'Implicit rho' settings make exactly, but to be safe it might be best to always explicitly enter the value yourself. Also keep in mind that you cannot enter the same value of $f$ into both applications and expect the same result. As to which sample size numbers are 'correct': that will depend on the true value of $\rho$...

$\endgroup$
1
  • 1
    $\begingroup$ +1. GPower's developers should probably improve their interface, it's not the first time I see questions/confusion created by these hidden options. Note that the "determine" button gives even more hidden options. On the question of what the different options do exactly, this other related discussion is certainly relevant: stats.stackexchange.com/questions/520090/… $\endgroup$
    – J-J-J
    Nov 8, 2023 at 11:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.