Additional Sum of Squares Table in R - how in the world do we interpret the F statistics?

In R when you run anova on a multiple regression you get a table like this

Analysis of Variance Table

Response: y
Df  Sum Sq Mean Sq  F value    Pr(>F)
x1         1    9.56    9.56   2.4237   0.13160
x2         1   28.24   28.24   7.1591   0.01273 *
x3         1 1201.42 1201.42 304.6223 7.081e-16 ***
Residuals 26  102.54    3.94



I understand that the Sum of Squares for each row is the additional sum of squares explained by this predictor when added to the model in this order

$$SS(\beta_1|\beta_0)=9.56$$

$$SS(\beta_2|\beta_0,\beta_1)=28.24$$

And I know how the F statistics are calculated in the table $$F = \frac{SS(row)/df(row)}{MSE}$$

Further I get that the final row's F statistic is the F stat you would get if you were testing $$H_0: \beta_k=0 | \beta_0,\ldots,\beta_{k-1}$$

What I'm not clear about is how to interpret the F statistics in the other rows of the table. Because each of them use MSE from the full model in their calculation, so I don't see how to phrase the null hypothesis in these cases. Any help would be appreciated. I'm currently of the mind that only the F stat in the final row has a "meaningful" hypothesis test associated with it, and the others should be ignored, or they are approximations somehow.

• The *** is supposed to suggest more significant in some sense than just * (which itself is still supposed to indicate significant) but this gets you into the rabbit-hole of $p$-values, especially since you would get different results if you dropped x1 Mar 2, 2023 at 18:34

There is a misunderstanding here. The order of variables entered into the model does not change how each F-statistic is interpreted.

The F-statistic for x3 is also not testing if any of the variables in the model are significant. Each individual F-statistic is

$$F_{\beta_k} = \frac{MS_{\beta_k}}{MSE}$$

The hypothesis test you are thinking of, $$H_0: \beta_k =0, \forall k=1,2,3$$ is the model test, that is if any of the variables are significant.

$$F_{model} = \frac{MS_{model}}{MSE}$$

Where $$MS_{model} = \sum_1^k\frac{SS_k}{df_k}$$, which we can calculate here

$$F_{model} = \frac{1239.22}{3.94} = 314.52$$

Note that this should be the same F-statistic that you get from R when you get the summary from a multiple regression.

So then the hypothesis tests for each variable is

$$H_0: \beta_1 = 0 \\ H_1: \beta_1 \neq 0$$

This is testing if there are differences in any level of the variable. Sometimes this is called a group level test.

So for your interpretations here: There is no significant effect of x1. There is statistically significant effect of x2 at the $$p< 0.05$$ level. This gives some evidence of an effect. There is a statistically significant effect of x3 at the $$p < 0.0001$$ level. This gives strong evidence that there is an effect.

In my experience, the order that variables are entered into a model is rarely important to actual interpretations, in modern data sets that have large dimensionality.

A fun side note, but if the variables are "balanced" i.e. orthogonal categorical variables, the order of variables in the model makes no difference in the sum of squares.