The F-statistic with all predictors vs. with predictors excluded

The F-statistic formula is:

$$F=\frac{(TSS-RSS)/p}{RSS/(n-p-1)}$$

Where $$TSS$$ is total sum of squares and is equal to $$\sum_{i=1}^n(y_i-\bar{y})^2$$ and $$RSS$$ is the residual sum of squares and is equal to $$\sum_{i=1}^2(y_i-\hat{y}_i)^2$$.

$$p$$ is the number of predictors in the regression and $$n$$ is the sample size.

This F-statistic is testing the null hypothesis that:

$$H_0:\beta_1=\beta_2=\dots=\beta_p=0$$

Sometimes we want to test that a particular subset of $$q$$ coefficients are 0. The book I am learning from states the following. The null hypothesis is:

$$H_0:\beta_{p-q+1} = \beta_{p-q+2} = \beta_p = 0$$

where for convenience the variables chosen for omission are at the end of the list.

We fit a second model that uses all the variables except those last $$q$$. The residual sum of squares for that model is referred to as $$RSS_0$$. The appropriate F-statistic is then:

$$F=\frac{(RSS_0-RSS)/q}{RSS/(n-p-1)}$$

1) What exactly do they mean by "the variables chosen for omission are at the end of the list"? I don't understand how $$\beta_{p-q+1}$$ and $$\beta_{p-q+2}$$ specify which coefficients to remove (i.e. the $$q$$ coefficients).

2) What happens to $$TSS$$ in the numerator of the second F-statistic equation?

3) How does this report the partial effect of adding a variable to a model?

4) Why does the F-test of a model that leaves out coefficient $$j$$ while leaving all others in (i.e. $$q=1$$) exactly equal the $$\text{p-value}$$ and $$\text{t-statistic}^2$$ of coefficient $$j$$ in a model that leaves all coefficients in?

• For #2, TSS is the same no matter what predictors you use. Consider the formula: $\sum_i\big(y_i-\bar{y}\big)^2$. The predictors don't get considered.
– Dave
Apr 1, 2020 at 22:26

1) As said in the text, for the convenience (of writing). They can be any $$\beta$$s that you want test.
2) You can think in this way: for the first model, you have $$TSS - RSS$$; for the second model, you have $$TSS - RSS_0$$. So the deference is $$RSS - RSS_0$$
3) Because $$RSS - RSS_0$$ is the contribution of $$q$$ variables after other $$p-q$$ variables already in the model.
4) A) When $$q=1$$, both the F test described here and t-test you described test the same null hypothesis $$H_0: \beta_j = 0$$. B) Also you can check the relationship between t distribution and F distribution, you will find under certain conditions, $$t^2 = F$$. C) Can prove that square of t statistics = F statistics under this situation and you may find it from internet.