4
$\begingroup$

I need to calculate effect size for a mixed ANOVA design. I need it because I'm using GPower3 to calculate a rough sample size for a full study. I have pilot data. Based on this page:

http://www.xlstat.com/en/features/statistical-power-analysis-of-variance-anova.htm

and the GPower3 software, it looks like there are 3 ways to calculate an effect size:

  • eta²
  • use variances
  • use group means

Is there any rational for picking one over another? Since I have the pilot data, I guess I was just going to use the variances to calculate the effect size. The formula I have for that is:

f (effect size) = sqrt(Variance Explained / Variance Error)

Based on the summation rule:

Total Variance = Variance Explained + Variance Error

Is this correct?

If I have unequal group sizes, could I use the pooled variance formula for the Variance Error term?

I also noticed that this effect size formula using the variances is quite similar to the F statistic:

F = Variance Explained / Variance Error

Is this correct and/or is there a relationship between them?

Thanks in advance.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

I am not familiar with Gpower3 so this does not exactly address your question, but here is another approach that you may want to consider.

For power and sample size studies more complicated than simple 2 sample tests I prefer to use simulation rather than prepackaged routines. The basic steps are:

  1. Think through what your data will look like (structure, means, variances, distributions, ...)
  2. Think through how you will analyse the data
  3. Generate data based on current assumptions and analyze it
  4. Repeate step 3 many times and compute the percentage of times that the null is rejected (this is the power)
  5. Repeate steps 3 and 4 for different sets of assumptions

One of the advantages of this approach is that you know exactly what assumptions are being made and what the nature of the data and analysis is. The main disadvantage is that it takes a bit more work and time than just plugging in a few numbers to a canned routine.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi Greg! This is a very interesting idea. I would like to try it out. Are you aware of any examples in published articles that make use of this way of power analysis? $\endgroup$
    – jokel
    Commented Oct 30, 2012 at 18:33
  • $\begingroup$ I don't know of any specific articles. I plan to write one (probably for the R Journal) sometime, but have not had the time yet. $\endgroup$
    – Greg Snow
    Commented Oct 30, 2012 at 19:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.