2
$\begingroup$

In R when you run anova on a multiple regression you get a table like this

Analysis of Variance Table

Response: y
          Df  Sum Sq Mean Sq  F value    Pr(>F)    
x1         1    9.56    9.56   2.4237   0.13160    
x2         1   28.24   28.24   7.1591   0.01273 *  
x3         1 1201.42 1201.42 304.6223 7.081e-16 ***
Residuals 26  102.54    3.94        

           

I understand that the Sum of Squares for each row is the additional sum of squares explained by this predictor when added to the model in this order

$SS(\beta_1|\beta_0)=9.56$

$SS(\beta_2|\beta_0,\beta_1)=28.24$

And I know how the F statistics are calculated in the table $$F = \frac{SS(row)/df(row)}{MSE}$$

Further I get that the final row's F statistic is the F stat you would get if you were testing $$H_0: \beta_k=0 | \beta_0,\ldots,\beta_{k-1}$$

What I'm not clear about is how to interpret the F statistics in the other rows of the table. Because each of them use MSE from the full model in their calculation, so I don't see how to phrase the null hypothesis in these cases. Any help would be appreciated. I'm currently of the mind that only the F stat in the final row has a "meaningful" hypothesis test associated with it, and the others should be ignored, or they are approximations somehow.

$\endgroup$
1
  • $\begingroup$ The *** is supposed to suggest more significant in some sense than just * (which itself is still supposed to indicate significant) but this gets you into the rabbit-hole of $p$-values, especially since you would get different results if you dropped x1 $\endgroup$
    – Henry
    Mar 2, 2023 at 18:34

1 Answer 1

0
$\begingroup$

There is a misunderstanding here. The order of variables entered into the model does not change how each F-statistic is interpreted.

The F-statistic for x3 is also not testing if any of the variables in the model are significant. Each individual F-statistic is

$$F_{\beta_k} = \frac{MS_{\beta_k}}{MSE}$$

The hypothesis test you are thinking of, $H_0: \beta_k =0, \forall k=1,2,3$ is the model test, that is if any of the variables are significant.

$$F_{model} = \frac{MS_{model}}{MSE}$$

Where $MS_{model} = \sum_1^k\frac{SS_k}{df_k}$, which we can calculate here

$$F_{model} = \frac{1239.22}{3.94} = 314.52$$

Note that this should be the same F-statistic that you get from R when you get the summary from a multiple regression.

So then the hypothesis tests for each variable is

$$H_0: \beta_1 = 0 \\ H_1: \beta_1 \neq 0$$

This is testing if there are differences in any level of the variable. Sometimes this is called a group level test.

So for your interpretations here: There is no significant effect of x1. There is statistically significant effect of x2 at the $p< 0.05$ level. This gives some evidence of an effect. There is a statistically significant effect of x3 at the $p < 0.0001$ level. This gives strong evidence that there is an effect.

In my experience, the order that variables are entered into a model is rarely important to actual interpretations, in modern data sets that have large dimensionality.

A fun side note, but if the variables are "balanced" i.e. orthogonal categorical variables, the order of variables in the model makes no difference in the sum of squares.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.