0
$\begingroup$

I might be missing something obvious here. In linear regression, F statistic is defined as (explained variance / p) / mean squared error, where p is number of independent variables. When there is no relationship between independent and dependent variables, wouldn't explained variance close to zero and thus F statistic close to zero?

More specifically, I'm referring to page 76 of "an introduction to statistical learning with applications in R".

Thank you

$\endgroup$
1
  • $\begingroup$ Some of the explanation here and here may help $\endgroup$ – Glen_b Aug 2 '15 at 7:08
2
$\begingroup$

I believe this question is more or less addressed here: Intuitive explanation of the F-statistic formula?. The idea is that the regression mean square is an estimate of $\sigma^2$ when the null model holds, and so both the numerator and denominator of the $F$ statistic can roughly be expected to be close to the error variance when there is no relationship between the predictors and response.

$\endgroup$
1
$\begingroup$

@dsaxton has already given a good answer. Essentially, you can think of the F-test as:

 (error variance + regression effects) / (error variance)

The numerator is the explained variance or variance between groups. The denominator is the explained variance within groups. If our null hypothesis is correct, the regression effects should be close to zero and therefore the F-statistic should be close to 1. More mathematically, we can say under the null hypothesis, the expectation of ratio of the denominator and numerator is close to one.

Note that it's not the absolute value matters. So you can have a model with small explained variance, but if the unexplained variance is even smaller the overall ratio will be far away from 1. Therefore you can conclude that your regression effect is significant.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.