# Trying to understand the F-statistic

According to wikipedia, in regression problems, consider 2 models, the F-statistic is $$F=\frac{(RSS_1-RSS_2)/(p_2-p_1)}{RSS_2/(n-p_2)}$$, where $p_1,p_2$ are the degrees of freedom of the 2 models respectively, $n$ is the sample size.

I can't understand $F$ intuitively, since I see that $F$ seems to be ratio between RSE of the 2 models, and it is used to choose the model which better fits the data, right? How to interpret the numerator in $F$, $(RSS_1-RSS_2)/(p_2-p_1)$?

-

Let's take each portion separately.

$RSS_1 - RSS_2$ is a measure of which model has a smaller sum of squares; smaller sums of squares indicate a better model. However, sums of squares go up with sample size (other things being equal) so we need to scale it. There are different ways we could do this, but one way is by dividing by the degrees of freedom, which are closely linked to sample size, hence $p_2 - p_1$ in the denominator of the numerator.

But we don't just want to know if model 1 is better than model 2, we want to know how much better, so, again, we scale it by turning it into a proportion of the scaled RSS for model 2.

-
In overfitting, we also have smaller sums of squares, but it doesn't indicate a good fit, right? –  loganecolss Feb 17 '14 at 12:20
It does indicate a better fit, but it's a false fit - it won't generalize. –  Peter Flom Feb 17 '14 at 12:48

Rewriting that equation makes it more intuitive:

$$F=\frac{(RSS_1-RSS_2)/RSS_1}{(df_1-df_2)/df_2}$$

The df (degrees of freedom) for each model is n-k, where n is the number of data points and k is the number of parameters fit.

This form of the equation makes it easier to interpret. The F ratio compares the fractional decrease in sum-of-squares going from the simpler ("1") to the more complicated ("2") models as compared to the fractional decrease in degrees of freedom. When the relative decrease in sum-of-squares is much greater than the relative decrease in df, the F ratio is high, the P value is low, and you conclude that the evidence favors the more complicated model.

-