2
$\begingroup$

By null hypothesis is true I mean that the means of each group is the same at population level.

I am wondering whether the expected value would depart from 1 in the case that the variances of the groups are different from one another at population level, or if the groups are non-normally distributed at population level, or in any other case. If it does, why does this change the expected value?

I'm thinking of a One-way independent groups ANOVA with three groups, but wonder if the answer to this question would generalise beyond that.

$\endgroup$
5
$\begingroup$

Even if all the assumptions of ANOVA hold, the expected value of F isn't 1!

So no.

The expectation of an $F_{\nu_1,\nu_2}$ is $\nu_2/(\nu_2-2)$.

See here.

If you violate any of the assumptions, the distribution in general won't be distributed as $F$ and that expectation (already different from 1) is unlikely to hold either.

$\endgroup$
3
  • $\begingroup$ I'm a little confused by the expectation not being 1. I feel I am misinterpreting your answer at stats.stackexchange.com/a/96544/9162. If under the null hypothesis the numerator and denominator of the F-ratio are estimators of the population variance, what is the reason the expected value of F is not 1? Is it because they're biased estimators of the population variance? $\endgroup$ – user1205901 - Reinstate Monica Sep 3 '14 at 13:00
  • $\begingroup$ My answer that you link to there says "Typically close to 1" (which is true). That's a little different from "has expected value equal to 1" **, because "expected value" has a particular meaning, and doesn't come out to be 1, but is always a bit larger (though if the denominator d.f. is large, it will be very close to 1). The numerator and denominator in the F both have the same average ($\sigma^2$), but the average of the ratio is larger than the ratio of the averages. $\quad\quad$ **(especially given your use of the word 'guaranteed', which means we can't handwave away the difference) $\endgroup$ – Glen_b Sep 4 '14 at 0:03
  • 2
    $\begingroup$ That it will be greater than 1 is guaranteed by Jensen's inequality. $\endgroup$ – Glen_b Oct 3 '14 at 2:02
2
$\begingroup$

Here are my answers to your question

(1) If the assumptions of one way ANOVA are satisfied, would the expected F value be 1?

No. Suppose you have two $\chi^2$ distributed random variables with $n$ and $m$ degree of freedom i.e $MS_{(1)} \sim \chi^2_n$ and $MS_{(2)} \sim \chi^2_m$ then their ratio adjusted for the degree of freedom follows a F distribution with parameters $n$ and $m$ $$\frac{MS_{(1)}/n}{MS_{(2)}/m} \sim F(n,m)$$ with mean equals $\frac{m}{m-2}$.

In one way ANOVA, your $MS_{(1)}$ would be between group mean squared error and $MS_{(2)}$ would be your within group means squared error. In the case when there are many samples within each group, the expectation of your $F$ test statistics will be close to 1 but not equal to 1.

(2) What if the assumptions of ANOVA are NOT satisfied.

There are different ways to test equal-variance assumption such as Bartlett, Brown-Forsythe, Levene test. Levene test is probably the mostly used one. If the test returns a significant result, you can use Welch's ANOVA instead which calculates a ratio of sum of squared error weighted by group variances. (Of course, the formulation of Welch's ANOVA is much more complicated. But weighted ratio is the basic idea.)

The final Welch's F statistics will still follow a $F(n,m)$ distribution with $m$ equals some non-interger values.

Hope this is clear.

Peter

$\endgroup$
2
$\begingroup$

See this post for the mathematical proof of $E[F_{v1,v2}] = \frac{v_2}{v_2-2}$. This means that when all the assumptions are met, $E[F_{v1,v2}]$ approaches 1 as $v_2$ grows larger.

You can write an R function to see this empirically:

Make3Groups <- function(n){ ## note: n = n per group
  g1 <- data.frame(y=rnorm(n), x=rep("g1", n))
  g2 <- data.frame(y=rnorm(n), x=rep("g2", n))
  g3 <- data.frame(y=rnorm(n), x=rep("g3", n))
  d <- rbind(g1, g2, g3)
  d
}

n_iter <- 1000
set.seed(1)
## n = 2, v2 = 3, E[F] = 3
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(2)))[[1]]$"F value"[1])) ## 2.360747
## n = 5, v2 = 12, E[F] = 1.2
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(5)))[[1]]$"F value"[1])) ## 1.157286
## n = 34, v2 = 99, E[F] = 1.021
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(34)))[[1]]$"F value"[1])) ## 1.056753

Note that empirical results are sometimes off because the number of iterations is relatively small (1000).

To see the effect of heterogeneity of variance on expectations, you can play with different combinations of variances/standard deviations. I used $\sigma^2_i=1^2, 3^2, 10^2$ in the example below. This severe heterogeneity of variance apparently inflated $E[F]$ to 1.54 from 1.2 and in parallel type-I error rate to 0.106 from 0.05.

sd_vector <- c(1, 3, 10)
set.seed(1)
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(5, sd_vector)))[[1]]$"F value"[1])) ## 1.541211
mean(replicate(n_iter, summary(aov(y~x, data=Make3Groups(5, sd_vector)))[[1]]$"Pr(>F)"[1]<0.05)) ## 0.106
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.