Maybe it is best to do an ANOVA to see if there are any significant
differences among the three levels of the factor. If so, then
do ad hoc tests along with some method (such as Bonferroni) to
avoid 'false discovery'.
Unless I had some previous experience with similar data to feel
confident that the three levels all have the same variability, I'd use
oneway.test in R, which does not assume equal variances.
Example:
ctrl = rnorm(20, 50, 2)
trta = rnorm(20, 51, 2)
trtb = rnorm(20, 52, 2.2)
x = c(ctrl, trta, trtb)
g = rep(1:3, each=20)
boxplot(x~g, col="skyblue2")
oneway.test(x~g)
One-way analysis of means
(not assuming equal variances)
data: x and g
F = 4.1362, num df = 2.000, denom df = 37.975, p-value = 0.02371
So there are significant differences at the 5% level. It seems obvious that the control and treatment B must differ. Then we could do ad hoc
Welch t tests at the 2% level to see if we can distinguish between treatments A and B, and between control and treatment A.
In my fictitious example, we cannot say treatment a is significantly
different from the other two levels of the factor.
t.test(ctrl, trta)$p.val
[1] 0.06865211 # not signif
t.test(trta, trtb)$p.val
[1] 0.3030164 # not signif
t.test(ctrl, trtb)$p.val
[1] 0.007009848 # signif as surmised
There are procedures in R, other statistical software, and online
to suggest sample sizes for the three levels, given the size of
difference $\Delta$ we want to detect, the (common) SD $\sigma$ of the three groups, and the desired power. Most of these require equal
variability in the levels and equal sample sizes. However, it is
possible to relatively simple simulations in R to check whether
particular sample sizes give adequate power for given $\Delta$ and $\sigma_i.$
In the fictitious situation above we might use $\sigma_1 = \sigma_2 = 2, \sigma_3 = 2.2,$ and $\Delta = 1.$ We found that $n = 20$ was large enough for
the main ANOVA, but not large enough for the ad hoc tests (where smaller values of $\Delta$ may be required). So, as you suggest, we should focus on the power of the t tests.
By trial and error, it is easy to see that $n=80$ suffices to get power about 74% for a
Welch t test with $\Delta=1, \sigma_1=2, \sigma_2-2.2,$ and significance level $\alpha = 2\%.$ (And $n-100$ for power about 84%.)
pv = replicate(10^5, t.test(rnorm(80,50,2),rnorm(80,51,2.2))$p.val)
mean(pv <= .02)
[1] 0.74339
pv = replicate(10^5, t.test(rnorm(100,50,2),rnorm(100,51,2.2))$p.val)
mean(pv <= .02)
[1] 0.84551
The search can be hurried along by (a) using $10^4$ replications early on and (b) using standard formulas involving noncentral t distributions for pooled t tests which have slightly better power than do Welch t tests.
