Does R Automatically Calculate Single Degrees of Freedom?

Question

I would like to know whether R produces single degrees of freedom tests for a formula. Assume we have a model in R:

model = lm(x ~ a + b + c, data=mydata)  # augmented model

Is R only doing an omnibus test and comparing my augmented model to a null model with only an intercept:

null=lm(x ~ 1, data=mydata)  # compact model

or is it doing multiple single-degree-of-freedom tests, such as comparing:

model = lm(x ~ a + b + c, data=mydata) # augmented model

to all of the following compact models:

null1=lm(x ~ b + c, data=mydata) # compact model1
null2=lm(x ~ a + c, data=mydata) # compact model2
null3=lm(x ~ a + b, data=mydata) # compact model3

I am worried because I was taught to compare models with only a single degree of freedom between them, ie an augmented model that includes the parameter of interest against a compact model that excludes the parameter of interest. So, if I was interested in the effect of a, then I was taught to compare the augmented model with the compact model as follows:

model = lm(x ~ a + b + c, data=mydata) #augmented model
null = lm(x ~ b + c, data=mydata)      #compact model
anova(null, model)                    # single degree-of-freedom comparison between augmented model and compact model.

But this latter approach isn't very often taught, particularly when one is using R, though Bodo Winter seems to be an exception [EDIT: I've realised that the Winter tutorial is for random effects using lmer(), which does not automatically produce a p value, so this would explain why he teaches the model comparison approach in that context]. Is there any point in doing the comparison?

In other words, if R is doing the first thing (comparing to a null with just an intercept), then I think this is contrary to what I've been taught, but if the latter (comparing to multiple nulls, each test being a single degree of freedom comparison), then I don't think there is a problem.

Answer 1

From your elaboration, it appears you want to know what the F statistic means when you summary(lm(x~a+b+c))?

Basically, it tells you the difference between the model x~a+b+c compared to the intercept only model x~1. That is not the same as comparing full vs reduced models. To compare full vs reduced models you are removing only one variable and seeing if that affects the sums of squares explained by the new model. On this case, anyone of a, b, or c could be having an effect. But the point is just look at the t values and you can see which one is significant--within the model anyway.

Or maybe that's exactly what you want, to just compare your model against the intercept only model, which is what it's doing; but a single significant t test would tell you that the F test would therefore have to be significant anyway.

Just to show you that the F statistic is the same: run the following code (I have generated a bunch of random uniform variables to make a regression, but you can replace it with your own variables and it should be the same)

    > a<-runif(100,1,3); b<-runif(100,1,2); c<-runif(100,1,2);summary(lm(a~b+c))

    Call:
    lm(formula = a ~ b + c)

    Residuals:
     Min      1Q  Median      3Q     Max 
     -0.8875 -0.5368 -0.1291  0.5712  1.0798 

    Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
    (Intercept)  2.09554    0.48939   4.282 4.36e-05 ***
    b           -0.13188    0.21375  -0.617    0.539    
    c           -0.01779    0.22475  -0.079    0.937    
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

    Residual standard error: 0.5995 on 97 degrees of freedom
    Multiple R-squared:  0.003936,  Adjusted R-squared:  -0.0166 
    F-statistic: 0.1916 on 2 and 97 DF,  p-value: 0.8259

Notice the F statistic and how many degrees of freedom it has

    > anova(lm(a~1),lm(a~b+c))
    Analysis of Variance Table

    Model 1: a ~ 1
    Model 2: a ~ b + c
     Res.Df    RSS Df Sum of Sq      F Pr(>F)
1        99 35.003                           
2        97 34.865  2   0.13776 0.1916 0.8259

Same F, same degrees of freedom.

Answer 2

The two approaches (the t and the partial F-with-1-df) are equivalent as long as the t is two-tailed.

There's no additional benefit to the F (it never tells you anything you couldn't get from the t-test), except for the fact that it generalizes - e.g. to testing more than one variable at a time, and to the general linear hypothesis.