If the omnibus F-test for testing is not significant, is it still possible for the pairwise comparison for testing to be significant? Continuous outcome data are collected for three independent groups of equal size (i.e., all group n’s are equal). A linear regression analysis is performed to model the group means. More specifically, indicator variables are created for each of the three groups, and two of the three indicator variables are included in a regression model. Alternatively, all three indicator variables could be included in a no- intercept model. Assume group 1 has the lowest observed mean and group 3 has the highest observed mean.
Design and carry out a simulation experiment to answer A and B below:
A) What value for the observed mean for group2 would result in the minimal value for the omnibus F-test for testing: H0: μ1 = μ2 = μ3? That is, would the F statistic be smallest if ...
• the observed mean for group 2 is exactly equal to either the observed mean for group 1 or the observed mean for group 3?
• the observed mean for group 2 is exactly equal to the average of the observed means for group 1 and group 3?
or
• the observed mean for group 2 is some other value (but remember it must lie between the observed means for group 1 and group 3)?
B) What value for the observed mean for group2 would result in the maximal value for the omnibus F-test for testing: H0: μ1 = μ2 = μ3?
C) If the omnibus F-test for testing: H0: μ1 = μ2 = μ3 is not significant, is it still possible for the pairwise comparison for testing H0: μ1 = μ3 to be significant? Justify your answer mathematically (i.e. not with simulation).
D) Justify your answer to (A) or (B) mathematically.
I am most confused with the part C for the mathematical solution, and also for A and B, my simulation somehow give me the different answer compare with the theoretical form as below. theoretically seems like the mean of group 1 and 3 make the F statistic the smallest, but simulation show when group2 equal the group1 give the F statistics the smallest value. This problem have been bother me for quite a long time without finding the satisfied solutions, any help would be much appreciated!
set.seed(1)
res <- NULL

for(i in 1:1000) {
  
  #Case i
  group1 <- rnorm(50, 5, 1)
  group2 <- rnorm(50, 5, 1)
  group3 <- rnorm(50, 15, 1)
  
  group <- rep(1:3, each = 50)
  y <- c(group1, group2, group3)
  mod <- anova(lm(y ~ factor(group)+0))
  f1 <- mod$`F value`[1]
  
  #case ii
  group1 <- rnorm(50, 5, 1)
  group2 <- rnorm(50, 7, 1)
  group3 <- rnorm(50, 15, 1)
  
  group <- rep(1:3, each = 50)
  y <- c(group1, group2, group3)
  mod <- anova(lm(y ~ factor(group)+0))
  f2 <- mod$`F value`[1]

  #case iii
  group1 <- rnorm(50, 5, 1)
  group2 <- rnorm(50, 10, 1)
  group3 <- rnorm(50, 15, 1)
  
  group <- rep(1:3, each = 50)
  y <- c(group1, group2, group3)
  mod <- anova(lm(y ~ factor(group)+0))
  f3 <- mod$`F value`[1]
  
  #Case iv
  group1 <- rnorm(50, 5, 1)
  group2 <- rnorm(50, 15, 1)
  group3 <- rnorm(50, 15, 1)
  
  group <- rep(1:3, each = 50)
  y <- c(group1, group2, group3)
  mod <- anova(lm(y ~ factor(group)+0))
  f4 <- mod$`F value`[1]
  
  res <- rbind(res, c(f1, f2, f3, f4))
}

apply(res, 2, mean)


u1<-5
u3<-15
u2<-seq(5,15,by=0.1)
umean<-(20+u2)/3
fstat<-(u1-umean)^2+(u2-umean)^2+(u3-umean)^2
plot(u2,fstat)

 A: For your first set of problems (A and B), your attempt to mimic a classic ANOVA in R is leading you astray. If you look at the details of a mod returned by your code, you will see that the F-test is based on 3 degrees of freedom for the numerator. An F-test with 3 groups, however, should only have 2 degrees of freedom for the numerator.
What's going on?
When you write:
mod <- anova(lm(y ~ factor(group)+0))

you are forcing a linear regression through the origin. That constraint gives you the (incorrect) extra degree of freedom. Think about what that forcing through the origin also does to sums of squares and mean squares, and to the numerator of the F-statistic. In ANOVA, nothing is tied to the origin and only differences of individual groups from the mean of the groups contribute to the numerator of the F-statistic.
R is smart enough to take a standard linear model with a categorical predictor, without forcing through the origin, and provide the corresponding F-test for ANOVA. Try just removing the +0 from your 4 calls to lm() and see what happens.
So your mathematical justification for the results of A and B, implicit in the last 6 lines of your code, is correct. Just convert the code to the corresponding general equations, and use calculus to find the minimum as a function of u2. The lowest ANOVA F statistic with 3 groups is when the middle group is at the mean of the extreme groups, as you suspected.
With respect to problem C, use the results of problems A and B and D. Consider what happens if the middle group is at the mean of the other 2 groups, the overall F test isn't quite significant, but you only want to compare the top against the bottom group.
