One of the basic figures you get when running multiple linear regression using almost any off-the-shelf software is the F statistics. However, I cannot recall one situation, where the F value was low enough that I could say, the ratio of model MSE and sample variance was not significant. I do understand that the figure makes sense if we compare two competing models, but the way it is usually reported by most software is by measuring the relative decrease in variance. Maybe I am overlooking something, but why report the F statistic at all?

  • $\begingroup$ That they're often significant wouldn't be too surprising, since people try to avoid situations where they're rare (such as doing power studies before doing experiments). Nevertheless, I've seen many insignificant F's. If you always see them you're probably in a situation where hypothesis testing is less useful than things like point estimates and confidence intervals. $\endgroup$ – Glen_b Oct 10 '14 at 8:59
  • $\begingroup$ I do a lot of time series analysis, so I suppose this is a situation where they are less useful. What exactly is the domain where you see many insignificant F's? $\endgroup$ – Nils Oct 10 '14 at 9:10
  • $\begingroup$ One example: Student psych experiments where they simply can't get the sample size they need for the sort of effect size they expect (or, not infrequently, where the effect is just a lot smaller than they'd hoped). I've helped analyze a bunch of them. $\endgroup$ – Glen_b Oct 10 '14 at 9:18

The F-test may give you some useful information in some cases. For example, sometimes we find that according to the t-tests the regressors are individually not significant, while the F-test rejects the null that all the regressors (except the intercept) are jointly not significant. This may be a sign of multicollinearity among the regressors, which will lead to higher standard errors of parameter estimates and larger confidence intervals and should be somehow addressed (for example, creating a proxy variable that is a combination of some of the regressors).

| cite | improve this answer | |
  • $\begingroup$ Thanks for the answer. I see your point. I did not think of multicollinearity. However, even in this extreme case, wouldn't it be enough to look at the R2 to become suspicious? $\endgroup$ – Nils Oct 10 '14 at 8:57
  • $\begingroup$ Yes, a high coefficient of determination in a model with individually not significant variables would raise the same suspicion. But the R2 would increase anyway as you add more and more variables even if there is no collinearity among them and are not significant, so it may be better to look at the F-test. $\endgroup$ – javlacalle Oct 10 '14 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.