# How to find P(X > F) in ANOVA F-test?

(Apologies for the ASCII tables, Stackexchange doesn't allow HTML tables and since I'm not supposed to link to an image, this is the only way I know of showing the data)

I'm learning about ANOVA F-testing and stumbled upon this problem:

There exists the following set of data regarding four teaching methods and the scores of students who were subject to each teaching method:

Method 1  Method 2  Method 3  Method 4
----------------------------------------
65        75        59        94
87        69        78        89
73        83        78        80
79        81        62        88
81        72        83
69        79        76
90
------------------------------------


$\bar{x} = 75.67$       $78.43$          $70.83$         $87.75$
$s = 8.17$          $7.11$            $9.58$           $5.80$

Where $\bar{x}$ is the mean of all scores for each teaching method and $s$ is the standard deviation for each method.

After conducting an ANOVA, I derive the following ANOVA table:

Source     | Deg. Freedom |    SS     |    MS     |   F  |
-------------------------------------------------------------
Treatment  |     3        | 712.59    |  237.53   | 3.77 |
Error      |     19       | 1196.63   |  62.98    |  -   |
Total      |     22       | 1909.22   |    -      |  -   |


Now the question is: Test a level $\alpha = 0.05$ The null hypothesis is that there is no difference in mean achievement for the four teaching techniques.

So to restate:

$H_0$: Teaching technique does not have an influence on mean achievement of students
$H_1$: Teaching technique does have an influence

I work out the critical f value to be $f_{3,19;0.95}$ from an F-distribution table to be $3.1274$

The next step is what I don't understand.

We can claim that the teaching technique does have an influence on the mean achievement of the students (with less than 5% chance of being wrong)

The associated p-value is:

$p = P(X>3.77) = 0.0281$

and indeed, p < $0.05$ (hence reject of $H_0$)

But now where is this $0.0281$ from? It looks to be the probability that X > 3.77. If I'm not wrong, in this case $X ~ F_{3,19} = 3.1274$ (as calculated before) and so the p value should be the probability that 3.1274 is greater than 3.77. Now how can 3.1274 ever be greater than 3.77?

• Think about it: the chance that $X$ exceeds 3.77 (the p-value) must be less than the chance that $X$ exceeds 3.1274 (equal to 0.05 by construction). "The probability that 3.1274 is greater than 3.77" does not make any sense. If you're still unclear about this, draw a sketch of the PDF of the F-distribution and highlight the areas under the curve that correspond to (a) the rightmost .05 of the total area and (b) the area to the right of 3.77. – whuber Nov 1 '11 at 3:38
• Hmm, ok, I think I see. I graphed the F-distribution $F_{3,19}$ on an online graph generator. So is the P value the area under the graph to the right of 3.77 or is it the PDF value of the graph at 3.77? – Arvin Nov 1 '11 at 4:45
• @Arvin - The p-value is the area to the right of 3.77; with PDFs, we only look at areas. – Karl Nov 1 '11 at 5:01
• Ok, thanks for that. Now how do we calculate the area to the right of 3.77? Is there a formula that i'm missing? – Arvin Nov 1 '11 at 5:10
• I realize that this is not the subject of your question, but in the way you have restated your hypothesis, do you notice that you have gone from testing for any statistical connection to presuming a causal connection between the two variables? – rolando2 Nov 1 '11 at 7:58

The answer can not be retrieved without the aid of computation.

If you look at the F distribution table for F(3,19), you'll see that (for F(3,20):

...

.050 | 3.10

.025 | 3.86

...

Which means 0.025 < p < 0.05.

I'm guessing that they "cheated" with MATLAB or something.

• Indeed they did cheat with MATLAB (Answer actually says that MATLAB gave them the answer). So I'm guessing I won't ever be asked to find a p-value in exact – Arvin Nov 1 '11 at 10:50

Since the test statistics follows F-ratio distribution, the $\mathbb{P}(X>z)$ is known in closed form.

Let $X \sim F(n,m)$. Then $$\mathbb{P}(X > z) = \tilde{B}_{\frac{m}{m+ n z}}\left( \frac{m}{2}, \frac{n}{2} \right)$$

For $n=3$ and $m=19$ and $z=3.77$, Wolfram Alpha produces the answer of $0.0280768$.

• What is B squiggly bar? – Arvin Nov 1 '11 at 5:56
• @Arvin $\tilde{B}$ stands for incomplete beta function, divided over the complete beta function, known as beta regularized – Sasha Nov 1 '11 at 12:33
• BTW, most people would not characterize the incomplete Beta function as a "closed form" except for special values of $z$. It's an integral without a closed form expression in terms of standard functions; its calculation usually proceeds by numerical quadrature or computing sums (of potentially arbitrary length). – whuber Nov 1 '11 at 13:48
• @whuber It's a matter of definition. If you have that special functions routinely available to you on a calculator, you would be less inclined to single it out from elementary functions. Beta regularized is implemented in Mathematica, as well as in R (link). – Sasha Nov 1 '11 at 14:23
• Mathematica's help suggests BetaRegularized is evaluated by quadrature, not exactly. It can be important to be aware of and understand the difference, especially from a computing standpoint, so I think this is more than just a "matter of definition." Regardless, in the present context it would seem that "evaluate an incomplete Beta integral" was not what the OP meant by "missing some formula," because it's unlikely to be any more helpful than telling him to "evaluate the CDF of the F distribution" (which is commoner in software than incomplete Beta integrals, although they are equivalent). – whuber Nov 1 '11 at 14:29