The 'power and sample size' procedures for one-way ANOVAs usually give answers for models in which the number of
replications at each level of the factor is equal. I suppose this is for simplicity and because the most efficient use of
resources (to pay for replications) is usually a 'balanced' design with equal number of replications per level.
However, in some circumstances it is appropriate or necessary
to have unequal numbers of replications for various levels.
Examples: (a) You may be testing three new drugs against a current standard drug and want more replications for the standard drug for better power in ad hoc tests of each new drug against the standard.
(b) Replications at some levels of the factor may be much
more expensive than other levels.
When choosing numbers of replications for an ANOVA, it
is worthwhile to anticipate that one may find that some differences are significant. Then it is a good idea to have sufficient power for useful ad hoc tests.
If you go sufficiently deeply into the formulas for power against different patterns of alternatives, you may be able
to get exact results for power with differing numbers of replications. Perhaps more simply, you can get reasonably
accurate power values by simulation based on unbalanced designs; I show one such simulation below.
Suppose you have an experiment with normal samples with population means $50, 54, 57,$ standard deviations $4, 6, 7$ and sample sizes $20, 15, 15.$ Then we show the R procedure oneway.test, which does not assume equal variances for the levels. You might
have a dataset similar to the fictitious one below.
set.seed(1225)
x1 = rnorm(20, 50, 4)
x2 = rnorm(15, 54, 6)
x3 = rnorm(15, 57, 7)
x = c(x1, x2, x3)
g = rep(1:3, c(20,15,15))
oneway.test(x ~ g)
One-way analysis of means
(not assuming equal variances)
data: x and g
F = 11.774, num df = 2.000, denom df = 27.597, p-value = 0.0002009
So, this particular dataset to the specifications above shows significance at below the 1%. A simulation of 100,000 datasets
will give a good idea of the power of such a design. [Notice that denom df < 47 above, because the three sample variances differ.]
set.seed(2021)
m = 10^5; pv = numeric(m)
g = rep(1:3, c(20,15,15))
for (i in 1:m) {
x1 = rnorm(20, 50, 4)
x2 = rnorm(15, 54, 6)
x3 = rnorm(15, 57, 7)
x = c(x1, x2, x3)
pv[i] = oneway.test(x~g)$p.val
}
mean(pv <= 0.05)
[1] 0.90488 # aprx power
2*sd(pv <= 0.05)/sqrt(m)
[1] 0.001855511 # aprx 95% margin of sim error
The power of oneway.test at the 5% level of significance for this design is about $0.905 \pm 0.002.$
Notes: (1) I do not know of an exact formula for the power
of oneway.test, which has different denominator degrees
of freedom for each dataset, based on the observed variances of the various levels. See R documentation.
(2) A for-loop is not the most elegant way to program
the simulation in R, but it seems to be among the easiest for
beginners to understand.
pwrpackage for finding power for a range of pre-established scenarios (unequal vs. equal sample sizes). For thenparameter you give the average number of observations in each group, and very importanly, for the parameterf, you calculate correctly the effect sizef, which is different if the group sample sizes will be balanced or unbalanced. I explain how to calculate correctly the effect size here: stats.stackexchange.com/a/637646/164936 . $\endgroup$