I was not able to reproduce the results you got from WebPower using the pilot data you supplied. I was able to reproduce your R code however. You are correct that you can't use the $\eta^2$ for Cohen's f, but $f^2 = \frac{\eta^2}{1-\eta^2}$ 1. "However, how should I compute the effect size from the pilot study" - use the $\eta^2$ from the pilot study. 2. "Why are there interaction effect sizes, i.e, the effect size for group x vs group y?" Those are the effect sizes for the pair-wise comparisons (if you were using a t-test or a TukeyHSD) ``` r require(dplyr) require(reshape2) pilot <- data.frame(option1 = c(6.3, 2.8, 7.8, 7.9, 4.9), option2 = c(9.9, 4.1, 3.9, 6.3, 6.9), option3 = c(5.1, 2.9, 3.6, 5.7, 4.5), option4 = c(1.0, 2.8, 4.8, 3.9, 1.6)) pilot2 <- pilot %>% reshape2::melt(value.name = "y") %>% dplyr::rename("option" = "variable") lm1 <- lm(y ~ option, data = pilot2) aov1 <- aov(lm1) means <- apply(pilot, 2, mean) vs <- apply(pilot, 2, var) # cohen's f for overall anova # eta^2 = SSR / SST eta.sq <- anova(lm1)$`Sum Sq`[2] / sum(anova(lm1)$`Sum Sq`) f <- sqrt(eta.sq / (1-eta.sq)) # cohen's d for pairwise d <- abs(means[c(1,1,1,2,2,3)] - means[c(2,3,4,3,4,4)]) / sqrt(((5-1)*vs[c(1,1,1,2,2,3)] + (5-1)*vs[c(2,3,4,3,4,4)])/ (5+5)) names(d) <- c("1-2", "1-3", "1-4", "2-3", "2-4", "3-4") require(pwr) # with 5 samples, we have the power to detect effect size f = 0.835 # i.e. with only 5 samples, we need a large effect to detect pwr::pwr.anova.test(k = 4, n = 5, sig.level = 0.05, power = 0.80) #> #> Balanced one-way analysis of variance power calculation #> #> k = 4 #> n = 5 #> f = 0.8352722 #> sig.level = 0.05 #> power = 0.8 #> #> NOTE: n is number in each group # since we have a really large effect in the pilot for f = 1.2, # we only need 3 per group to detect with 80% power pwr::pwr.anova.test(k = 4, f = 1.2414, sig.level = 0.05, power = 0.80) #> #> Balanced one-way analysis of variance power calculation #> #> k = 4 #> n = 2.950833 #> f = 1.2414 #> sig.level = 0.05 #> power = 0.8 #> #> NOTE: n is number in each group ```