For testing whether a group of multiple coefficients ($\beta_{q+1}...\beta_p$) in a linear regression of the form $Y = \beta_0 + \beta_1 X_1\dots+\beta_{q}X_{q}+\beta_{q+1}X_{q+1}\dots+\beta_{p}X_{p}+\epsilon$ (with $\epsilon$ ~ $N(0,\sigma^2)$) are all simultaneously zero using the F test, we would test $H_0: {\beta_{q+1}...\beta_p = 0}$ vs the alternative where at least one of ${\beta_{q+1}...\beta_p \neq 0}$.

This is usually done with the following $F$ statistic: $\frac{(\hat\sigma_{null}^2-\hat\sigma_{full}^2)/(p-q)}{\hat\sigma_{full}^2/(n-p-1)}$ ~ $F_{p-q,n-p-1}$ where $\hat\sigma_{null}^2$ is the in sample mean sum of square residuals when fitting under the null hypothesis and $\hat\sigma_{full}^2$ is the same for the alternative.

If the null hypothesis is actually false, when trying to fit the null model (which ignores $X_{q+1}\dots X_p$), we would basically be lumping $\beta_{q+1}X_{q+1}\dots+\beta_{p}X_{p}$ with the noise and the form of this combination is not guaranteed to be Gaussian as assumed of the noise in the null model. If so, I believe that $\frac{n(\hat\sigma_{null}^2-\hat\sigma_{full}^2)}{\sigma^2}$ would no longer be $\chi_{p-q}^2$ as assumed in the $F$ statistic? If so, does this number mean anything, or does it simply serve the purpose of giving us a high number to reject the null hypothesis?

  • $\begingroup$ F-tests in linear regression simply test the possibility that at least one of the parameters in a multiple regression are significant. $\endgroup$ – user234562 May 2 '20 at 16:30
  • 1
    $\begingroup$ @user332577 Please refer to my liked post to see why the F-test doesn't say if any of the individual t-tests will give statistically significant results: stats.stackexchange.com/a/491239/247274. $\endgroup$ – Dave Oct 21 '20 at 22:14

The F-distribution used in hypothesis testing is not the sampling distribution of the F-ratio computed in your sample. It is the sampling distribution of the F-ratio under the null hypothesis. We simply compare our computed F-ratio to the sampling distribution under the null hypothesis to assess whether our assumption about the population from which the sample was drawn (i.e., the null hypothesis) is compatible with the data actually observed.

The true sampling distribution of the F-ratio in the population (i.e., under the truth, the alternative hypothesis, as you proposed) is a legitimate mathematical object, but it doesn't help in assessing the plausibility of the null hypothesis. So even if the sampling distribution of the F-ratio in the true population isn't an F-distribution, it's irrelevant to using the F-test to test the null hypothesis.

One time we are interested in the sampling distribution of a test statistic under an alternative hypothesis is when performing power analysis, in which case the true sampling distribution of the F-ratio helps us figure out the probability of drawing a sample that would lead us to reject the null hypothesis. In null hypothesis testing, however, typically only the sampling distribution under the null hypothesis is used.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.