If I have a logistic regression model with 3 predictors, $x_1$, $x_2$, $x_3$, and then I remove $x_3$ from my model (left with only $x_1$ and $x_2$), are those models nested? And therefore I can use the likelihood ratio test ($\chi^2$ with 1 df in this case) to test if $x_3$ is necessary to keep in the model. Or are these models not nested because they do not include the exact same variables?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Yes those models are nested. Quoting wikipedia's Statistical_model entry: Nested models are models that can be obtained by restricting a parameter in a more complex model to be zero. which is exactly what is happening when you "remove" $x_3$.
Additionally: Assuming you want to use Akaike information criterion (AIC - probably one of the most commonly used Information Criteria using likelihood ratios), you don't even need to worry about that. AIC works with non-nested models also.
-
$\begingroup$ Okay great, thanks! Wouldn't the log-likelihood test be a better test than AIC if my models are nested, though? Since using log-likelihood I can say that the models are significantly different, whereas with AIC I cannot say one is necessarily significantly different than the other? $\endgroup$– MarcusCommented Apr 19, 2013 at 18:41
-
$\begingroup$ Not necessarily; I think it is a matter of the methodology and the assumptions you wish to accept. Both tests share a significant number of similarities. Personally I feel that the AIC's Kullback–Leibler divergence-based theoretical assumptions are more sensible; eg. in contract with a standard LR test, AIC is not based on the assumption that at least one of the models compared is correct; additionally for a "small" sample size the asymptotic approximation that the LR statistic follows are not guaranteed. (AIC isn't perfect either, broadly speaking it usually over-fits.) You pick your poison. $\endgroup$ Commented Apr 19, 2013 at 22:49