14
$\begingroup$

Both the likelihood ratio test and the AIC are tools for choosing between two models and both are based on the log-likelihood.

But, why the likelihood ratio test can't be used to choose between two non-nested models while AIC can?

$\endgroup$
  • $\begingroup$ Akaike himself thought that AIC was useful for comparing non-nested models. See his quote that I referenced in response to the post here. $\endgroup$ – JonesBC Mar 18 '18 at 3:57
21
$\begingroup$

The LR (likelihood ratio) test actually is testing the hypothesis that a specified subset of the parameters equal some pre-specified values. In the case of model selection, generally (but not always) that means some of the parameters equal zero. If the models are nested, the parameters in the larger model that are not in the smaller model are the ones being tested, with values specified implicitly by their exclusion from the smaller model. If the models aren't nested, you aren't testing this any more, because BOTH models have parameters that aren't in the other model, so the LR test statistic doesn't have the asymptotic $\chi^2$ distribution that it (usually) does in the nested case.

AIC, on the other hand, is not used for formal testing. It is used for informal comparisons of models with differing numbers of parameters. The penalty term in the expression for AIC is what allows this comparison. But no assumptions are made about the functional form of the asymptotic distribution of the differences between the AIC of two non-nested models when doing the model comparison, and the difference between two AICs is not treated as a test statistic.

I'll add that there is some disagreement over the use of AIC with non-nested models, as the theory is worked out for nested models. Hence my emphasis on "not...formal" and "not...test statistic." I use it for non-nested models, but not in a hard-and-fast way, more as an important, but not the sole, input into the model building process.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ @Carl - the elaboration is in the two comments immediately prior to the comment you quote. I think you should take gung's advice - post a question and answer it. That's a fair thing to do in these circumstances, and others have done similarly for "reference questions" as well. Having just gone through your answer, I'd upvote it. $\endgroup$ – jbowman Oct 3 '18 at 0:26
  • $\begingroup$ I took the advice and new question and answer are here. BTW, I upvoted your question (and the accepted answer) because it made me think, and not because I completely agree with it. My problem is that the assumption that non-nested models can be compared by AIC is only true when lots of other usually ignored conditions are met. $\endgroup$ – Carl Oct 3 '18 at 0:51
9
$\begingroup$

The derivation of AIC as an estimator of Kullback-Leibler information loss makes no assumptions of models being nested.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ But Akaike did make the assumption that the models were being constructed on the same data. $\endgroup$ – DWin Mar 10 '15 at 0:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.