Use of likelihood ratio test/ANOVA for significance testing I've read that likelihood ratio tests comparing two models (one with and without a predictor) should be performed to determine whether a variable of interest is statistically significant, rather than using the p-values for estimates of individual predictors from the summary() function of a linear model.
I've also read that this is only necessary when the model includes factors with more than two levels.
I am trying to find out whether the second statement is correct but have been unable to find out whether or not LRT/ANOVA is necessary for models with factors containing only two levels.
Please could anyone advise?
 A: You can test the nested models using either Wald or likelihood ratio testing. Wald would be the standard way to go with a linear model. The reduced model only has the continuous predictor, and then the full model has the continuous predictor plus the others. Your null is that the other predictors do not influence the outcome, and the alternative is that they do influence the outcome.
Wald and likelihood ratio methods test these hypotheses in somewhat different ways but more-or-less aim to justify the inclusion of additional predictors. The fit never decreases when you add predictors, but is the increase in fit worth the added complexity?
Wald compares the ratio of squared errors to an $F$-distribution (sound familiar from ANOVA?), while likelihood ratio compares the ratio of likelihoods to a $\chi^2$ distribution. I'm going from memory and might have missed some details, but these should look somewhat familiar.
$$\text{**Wald Test**}$$
$$\dfrac{(SSE_{reduced}-SSE_{full})/(n-p_{full})}{SSE_{reduced}/(p_{full}-p_{reduced})}\sim F_{n-p_{full}, p_{full}-p_{reduced}}$$
$$\text{**Likelihood-ratio Test**}$$
$$[LLik_{full} - LLik_{reduced}] \sim \chi^2_{\text{difference in parameter counts of the nested full and reduced models}}$$
A: It depends on what you're interested in testing. Let's consider a linear model with a two-category predictor $X$ and some response $Y$. The model is:
$y_{i} = \beta_{0} + \beta_{1}d_{i} + \varepsilon_{i}$
Where $d_{i}$ is an indicator/dummy variable which could be written as $d_{i} = \mathbb{I}(x=1)$. That is: It takes on the value 0 for one category, and the value 1 for the other category. You fit this in R using the command lm( y ~ x ).
The summary() command in R is going to test all of the individual coefficients, in this case $\beta_{0}$ and $\beta_{1}$. The anova() command (comparing the previous lm call with a second lm call lm( y ~ 1 )) will perform a test on all the coefficients of $X$. For the case of a two-category predictor, this is the same test as the individual test from summary().
Now consider a 3-category predictor. With three categories, we need two dummy variables, so the model is:
$y_{i} = \beta_{0} + \beta_{1}d_{1,i} + \beta_{2}d_{2,i} + \varepsilon_{i}$
We can run the same codes as before. Before, the individual test from summary() was (or rather, included) a test on $\beta_{1}$, which was equivalent with a test on the entire $X$ predictor. with the 3-category predictor, none of the individual tests are testing "all" of $X$. But the anova() test is assessing the "total effect" of $X$, so the tests are no longer the same.
If we consider four models:

*

*Full: $y_{i} = \beta_{0} + \beta_{1}d_{1,i} + \beta_{2}d_{2,i} + \varepsilon_{i}$

*Reduced 1b: $y_{i} = \beta_{0} + \beta_{2}d_{2,i} + \varepsilon_{i}$

*Reduced 1a: $y_{i} = \beta_{0} + \beta_{1}d_{1,i} + \varepsilon_{i}$

*Reduced 2: $y_{i} = \beta_{0} + \varepsilon_{i}$
The summary() output is comparing the Full model to Reduced models 1a and 1b (ignoring the test on the intercept). the anova() model is comparing the Full model to Reduced model 2. For a two-category predictor these will align, for three or more categories, they do not.
The individual tests may still be useful with 3+ categories, but it depends on what you're wanting to test.
You can see it in action with some examples. Change kk from 2 to 3, for instance. Notice that for kk <- 2 the p-value for the predictor in summary() will be the same as that from anova(), and if you square the t test statistic, you'll get the LRT test statistic. When you switch to kk <- 3, that will no longer be the case.
nn <- 10
kk <- 2

gg <- rep( LETTERS[1:kk], each=nn )

xx <- runif( nn*kk , 0, 10 )
yy <- rnorm( nn*kk, rep( c(10,12,14)[1:kk] , each=nn), 3 )


summary( lm( yy ~  gg ) )
anova( lm( yy ~  gg ),
       lm( yy ~ 1 ) )

