The F-statistic formula is:

$$ F=\frac{(TSS-RSS)/p}{RSS/(n-p-1)} $$

Where $TSS$ is total sum of squares and is equal to $\sum_{i=1}^n(y_i-\bar{y})^2$ and $RSS$ is the residual sum of squares and is equal to $\sum_{i=1}^2(y_i-\hat{y}_i)^2$.

$p$ is the number of predictors in the regression and $n$ is the sample size.

This F-statistic is testing the null hypothesis that:

$$ H_0:\beta_1=\beta_2=\dots=\beta_p=0 $$

Sometimes we want to test that a particular subset of $q$ coefficients are 0. The book I am learning from states the following. The null hypothesis is:

$$ H_0:\beta_{p-q+1} = \beta_{p-q+2} = \beta_p = 0 $$

where for convenience the variables chosen for omission are at the end of the list.

We fit a second model that uses all the variables except those last $q$. The residual sum of squares for that model is referred to as $RSS_0$. The appropriate F-statistic is then:

$$ F=\frac{(RSS_0-RSS)/q}{RSS/(n-p-1)} $$

1) What exactly do they mean by "the variables chosen for omission are at the end of the list"? I don't understand how $\beta_{p-q+1}$ and $\beta_{p-q+2}$ specify which coefficients to remove (i.e. the $q$ coefficients).

2) What happens to $TSS$ in the numerator of the second F-statistic equation?

3) How does this report the partial effect of adding a variable to a model?

4) Why does the F-test of a model that leaves out coefficient $j$ while leaving all others in (i.e. $q=1$) exactly equal the $\text{p-value}$ and $\text{t-statistic}^2$ of coefficient $j$ in a model that leaves all coefficients in?


1) As said in the text, for the convenience (of writing). They can be any $\beta$s that you want test.

2) You can think in this way: for the first model, you have $TSS - RSS$; for the second model, you have $TSS - RSS_0$. So the deference is $RSS - RSS_0$

3) Because $RSS - RSS_0$ is the contribution of $q$ variables after other $p-q$ variables already in the model.

4) A) When $q=1$, both the F test described here and t-test you described test the same null hypothesis $H_0: \beta_j = 0$. B) Also you can check the relationship between t distribution and F distribution, you will find under certain conditions, $t^2 = F$. C) Can prove that square of t statistics = F statistics under this situation and you may find it from internet.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.