In ANOVA, if the null hypothesis is true is the expected value of F guaranteed to be 1?

Question

By null hypothesis is true I mean that the means of each group is the same at population level.

I am wondering whether the expected value would depart from 1 in the case that the variances of the groups are different from one another at population level, or if the groups are non-normally distributed at population level, or in any other case. If it does, why does this change the expected value?

I'm thinking of a One-way independent groups ANOVA with three groups, but wonder if the answer to this question would generalise beyond that.

Answer 1

Even if all the assumptions of ANOVA hold, the expected value of F isn't 1!

So no.

The expectation of an $F_{\nu_1,\nu_2}$ is $\nu_2/(\nu_2-2)$.

See here.

If you violate any of the assumptions, the distribution in general won't be distributed as $F$ and that expectation (already different from 1) is unlikely to hold either.

Answer 2

Here are my answers to your question

(1) If the assumptions of one way ANOVA are satisfied, would the expected F value be 1?

No. Suppose you have two $\chi^2$ distributed random variables with $n$ and $m$ degree of freedom i.e $MS_{(1)} \sim \chi^2_n$ and $MS_{(2)} \sim \chi^2_m$ then their ratio adjusted for the degree of freedom follows a F distribution with parameters $n$ and $m$ $$\frac{MS_{(1)}/n}{MS_{(2)}/m} \sim F(n,m)$$ with mean equals $\frac{m}{m-2}$.

In one way ANOVA, your $MS_{(1)}$ would be between group mean squared error and $MS_{(2)}$ would be your within group means squared error. In the case when there are many samples within each group, the expectation of your $F$ test statistics will be close to 1 but not equal to 1.

(2) What if the assumptions of ANOVA are NOT satisfied.

There are different ways to test equal-variance assumption such as Bartlett, Brown-Forsythe, Levene test. Levene test is probably the mostly used one. If the test returns a significant result, you can use Welch's ANOVA instead which calculates a ratio of sum of squared error weighted by group variances. (Of course, the formulation of Welch's ANOVA is much more complicated. But weighted ratio is the basic idea.)

The final Welch's F statistics will still follow a $F(n,m)$ distribution with $m$ equals some non-interger values.

Hope this is clear.

Peter

Answer 3

See this post for the mathematical proof of $E[F_{v1,v2}] = \frac{v_2}{v_2-2}$. This means that when all the assumptions are met, $E[F_{v1,v2}]$ approaches 1 as $v_2$ grows larger.

You can write an R function to see this empirically:

Make3Groups <- function(n){ ## note: n = n per group
  g1 <- data.frame(y=rnorm(n), x=rep("g1", n))
  g2 <- data.frame(y=rnorm(n), x=rep("g2", n))
  g3 <- data.frame(y=rnorm(n), x=rep("g3", n))
  d <- rbind(g1, g2, g3)
  d
}

n_iter <- 1000
set.seed(1)
## n = 2, v2 = 3, E[F] = 3
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(2)))[[1]]$"F value"[1])) ## 2.360747
## n = 5, v2 = 12, E[F] = 1.2
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(5)))[[1]]$"F value"[1])) ## 1.157286
## n = 34, v2 = 99, E[F] = 1.021
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(34)))[[1]]$"F value"[1])) ## 1.056753

Note that empirical results are sometimes off because the number of iterations is relatively small (1000).

To see the effect of heterogeneity of variance on expectations, you can play with different combinations of variances/standard deviations. I used $\sigma^2_i=1^2, 3^2, 10^2$ in the example below. This severe heterogeneity of variance apparently inflated $E[F]$ to 1.54 from 1.2 and in parallel type-I error rate to 0.106 from 0.05.

sd_vector <- c(1, 3, 10)
set.seed(1)
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(5, sd_vector)))[[1]]$"F value"[1])) ## 1.541211
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(5, sd_vector)))[[1]]$"Pr(>F)"[1]<0.05)) ## 0.106