What is the meaning of an F value less than 1 in one-way ANOVA?

Question

What does it mean if the F value in one-way ANOVA is less than 1?

Remember the F-ratio is

$$\frac{\sigma^2+\frac{r\times\sum_{i=1}^t \tau_i^2}{t-1}}{\sigma^2}$$

Answer 1

The F ratio is a statistic. When the null hypothesis of no group differences is true, then the expected value of the numerator and denominator of the F ratio will be equal. As a consequence, the expected value of the F ratio when the null hypothesis is true is also close to one (actually it's not exactly one, because of the properties of expected values of ratios).

When the null hypothesis is false and there are group differences between the means, the expected value of the numerator will be larger than the denominator. As such the expected value of the F ratio will be larger than under the null hypothesis, and will also more likely be larger than one.

However, the point is that both the numerator and denominator are random variables, and so is the F ratio. The F ratio is drawn from a distribution. If we assume the null hypothesis is true we get one distribution, and if we assume that it is false with various assumptions about effect size, sample size, and so forth we get another distribution. We then do a study and get an F value. When the null hypothesis is false, it is still possible to get an F ratio less than one. The larger the population effect size is (in combination with sample size), the more the F distribution will move to the right, and the less likely we will be to get a value less than one.

The following graphic extracted from the G-Power3 demonstrates the idea given various assumptions. The red distribution is the distribution of F when H0 is true. The blue distribution is the distribution of F when H0 is false given various assumptions. Note that the blue distribution does include values less than one, yet they are very unlikely.

Answer 2

Your question in the title is an interesting question that crossed my mind today too. I just want to add a correction. The F-ratio is : $$\frac{MS_{treatment}}{MS_{residual}}=\frac{\frac{SS_{treatment}}{t-1}}{\frac{SS_{residual}}{t(r-1)}}$$ What you wrote is the $$\frac{E(MS_{treatment})}{E(MS_{residual})}$$ While the first fraction can be less than 1, the second fraction cannot be less than 1. But that's not a problem since it's a quotient of expectations.

Answer 3

Note that while values of the F statistic less than 1 can occur by chance when the null hypothesis is true (or near true) as others have explained, values close to 0 can indicate violations of the assumptions that ANOVA depends on. Some analysts will look at the area to the left of the statistic in the F-distribution as a p-value checking assumption violations. Some of the violations that lead to small F-stats include unequal variances, improper randomization, lack of independence, or just faking the data.

Answer 4

Doesn't it just mean that your treatment was almost completely ineffective?

Answer 5

The issue here is that hypothesis testing involves a null AND an alternative hypotheis, and therefore; the rejection region is determined by both hypotheses.

Consider a simpler example. If you are investigating a process that possibly has a MEAN of zero, but could not have a mean of less than zero, then you might be interested in performing the following test

\begin{equation} \begin{array}{c} H_{0}: \mu = 0 \ H_{1}: \mu > 0 \ \end{array} \nonumber \end{equation}

at a level alpha. Your rejection region of the null hypothesis is on the right of zero. It is not impossible for you to get a sample mean that is negative, albeit with a small probability. If you were to get a negative sample mean in your experiment, you would not question the veracity of the experiment.

Now consider your question. The reason that the rejection region for the F-statistic is on the right is because of the alternative hypothesis in the one-way ANOVA. You are testing the hypothesis that

\begin{equation} \begin{array}{c} H_{0}: \sum \tau_{i}^{2} = 0 \ H_{1}: \sum \tau_{i}^{2} \ne 0 \ \end{array} \nonumber \end{equation}

The null hypothesis dictates that you use the central F distribution, and the alternative hypothesis, forcing the distribution to the right when the alternative hypothesis is true means that all of the Type I error probability must be located on the right.

Is it possible for the test statistic to be less than one? When the null hypothesis is true, it is certainly possible; just as in the previous example where it was possible for the test statistic to be negative even if the MEAN of the data is zero.

Answer 6

After looking in a folder I haven't looked for years (like real folder and not computer folder) I found this paper which may be of interest for this question:

Voelkle, M. C., Ackerman, P. L., & Wittmann, W. W. (2007). Effect Sizes and F Ratios < 1.0. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 3(1), 35–46. doi:10.1027/1614-2241.3.1.35

The abstract says:

Standard statistics texts indicate that the expected value of the $F$ ratio is $1.0$ (more precisely: $N/(N-2)$) in a completely balanced fixed-effects ANOVA, when the null hypothesis is true. Even though some authors suggest that the null hypothesis is rarely true in practice (e.g., Meehl, 1990), $F$ ratios $< 1.0$ are reported quite frequently in the literature. However, standard effect size statistics (e.g., Cohen's $f$) often yield positive values when $F < 1.0$, which appears to create confusion about the meaningfulness of effect size statistics when the null hypothesis may be true. Given the repeated emphasis on reporting effect sizes, it is shown that in the face of $F < 1.0$ it is misleading to only report sample effect size estimates as often recommended. Causes of $F$ ratios $< 1.0$ are reviewed, illustrated by a short simulation study. The calculation and interpretation of corrected and uncorrected effect size statistics under these conditions are discussed. Computing adjusted measures of association strength and incorporating effect size confidence intervals are helpful in an effort to reduce confusion surrounding results when sample sizes are small. Detailed recommendations are directed to authors, journal editors, and reviewers.