Power for experimental design

Question

It's easy to calculate power for paired t-tests and ANOVA, but how do you calculate power for more complicated designs such as split plot designs or factorial designs?

Answer 1

First turn your anova into a regression. Then,

$$ N = \frac{2.84^2}{p(1-p)}\frac{\sigma^2}{MDE^2} $$

N is your sample size, p is the proportion getting the treatment, $\sigma$ is the standard deviation of the residuals, and MDE=the minimum detectable effect that you are powered for. 2.84 comes from alpha confidence of 95% and 80% power (when you've got a lot of degrees of freedom. With fewer degrees of freedom, you'll need to use the T distribution. Look at the reference for details.)

The more explanatory factors you've got in your experiment the more $\sigma$ will shrink. Knowing how much it will shrink is tricky.

At some point, analytical formulas collapse, and you're better off simply simulating your entire dataset, multiple times doing some sort of monte carlo, and fitting the desired model to each of the plausible datasets. Your power is the proportion of times you get the result you want. As with the analytical methods, you're only as good as your assumptions.

Source is from memory, roughly following this.

Answer 2

Once beyond the simple cases like t-tests I prefer to use simulations. When you do a simulation you control all the assumptions being made and can simulate for situations that may not be in the nice canned routines.

Here is an answer with an example of a simulation:

Simulation 1