2
$\begingroup$

If I have a logistic regression model with 3 predictors, $x_1$, $x_2$, $x_3$, and then I remove $x_3$ from my model (left with only $x_1$ and $x_2$), are those models nested? And therefore I can use the likelihood ratio test ($\chi^2$ with 1 df in this case) to test if $x_3$ is necessary to keep in the model. Or are these models not nested because they do not include the exact same variables?

$\endgroup$
2
$\begingroup$

Yes those models are nested. Quoting wikipedia's Statistical_model entry: Nested models are models that can be obtained by restricting a parameter in a more complex model to be zero. which is exactly what is happening when you "remove" $x_3$.

Additionally: Assuming you want to use Akaike information criterion (AIC - probably one of the most commonly used Information Criteria using likelihood ratios), you don't even need to worry about that. AIC works with non-nested models also.

$\endgroup$
2
  • $\begingroup$ Okay great, thanks! Wouldn't the log-likelihood test be a better test than AIC if my models are nested, though? Since using log-likelihood I can say that the models are significantly different, whereas with AIC I cannot say one is necessarily significantly different than the other? $\endgroup$
    – Marcus
    Apr 19 '13 at 18:41
  • $\begingroup$ Not necessarily; I think it is a matter of the methodology and the assumptions you wish to accept. Both tests share a significant number of similarities. Personally I feel that the AIC's Kullback–Leibler divergence-based theoretical assumptions are more sensible; eg. in contract with a standard LR test, AIC is not based on the assumption that at least one of the models compared is correct; additionally for a "small" sample size the asymptotic approximation that the LR statistic follows are not guaranteed. (AIC isn't perfect either, broadly speaking it usually over-fits.) You pick your poison. $\endgroup$
    – usεr11852
    Apr 19 '13 at 22:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.