Introduction to Statistical Learning defines the F-statistic as follows: $$\frac{(TSS - RSS) / p}{RSS / (n - p - 1)}$$ I am trying to interpret this formula intuitively - the numerator looks like the $ESS$ per regressor, and the denominator looks like the $RSS$ per observation. This does not seem like an apples to apples comparison; can anyone explain why it makes sense?
They also say that the expectation of the denominator is equal to the variance of the irreducible error if the linear model assumptions hold (I get this, as the denominator is really the Residual Standard Error, which is an unbiased estimator). They also say the numerator is equal to the variance of the irreducible error if the null hypothesis is true. Therefore the F-statistic will be close to 1 if the null hypothesis is true.
But if you let p = 1, that is, apply the F-statistic to single linear regression, it becomes:
$$\frac{TSS - RSS}{RSS / (n-2)}$$
According to the book, if the regressor has no explanatory power, the F-statistic should be close to 1. But if you imagine a data set where the coefficient on X is 0 (i.e. no explanatory power), $TSS$ will be equal to $RSS$, so the numerator and hence the F-statistic should be 0, not 1 as they claim. What is going on?
Furthermore, if you accept that the F-statistic is 1, as they claim, then $(TSS - RSS) = RSS / (n-2)$, which means $ESS = RSS / (n-2)$. If you think of the F-statistics as comparing explained vs. unexplained variation, this does not seem like a fair comparison since this decomposition makes it the ratio of TOTAL explained variation summed across ALL observations vs. the unexplained variation PER observation. Again, what am I missing?
I am just trying to make sense of it in a layman way, apologies if I am missing something obvious.