# Can AIC compare across different types of model?

I'm using AIC (Akaike's Information Criterion) to compare non-linear models in R. Is it valid to compare the AICs of different types of model? Specifically, I'm comparing a model fitted by glm versus a model with a random effect term fitted by glmer (lme4).

If not, is there a way such a comparison can be done? Or is the idea completely invalid?

-

It depends. AIC is a function of the log likelihood. If both types of model compute the log likelihood the same way (i.e. include the same constant) then yes you can, if the models are nested.

I'm reasonably certain that glm() and lmer() don't use comparable log likelihoods.

The point about nested models is also up for discussion. Some say AIC is only valid for nested models as that is how the theory is presented/worked through. Others use it for all sorts of comparisons.

-
My understanding is that lme4, by default uses REML where glm uses ML. They might be comparable if you made lmer use ML by setting REML = FALSE. – Russell S. Pierce Nov 29 '10 at 20:46
In addition to your Gavin's comment, it also depends what do you want to do with the model. Is the model for prediction or Thomas is looking for parsimony ? (I think) – suncoolsu Nov 29 '10 at 20:52
@drnexus: I don't think that is sufficient; you have to be sure that the same normalising constant is being applied in the log likelihood calculation. – Gavin Simpson Nov 29 '10 at 20:52
@suncoolsu: I'm not trying to use this model for prediction. @Gavin: How would I know about normalising constants? I confess my understanding of what R is doing behind the scenes is minimal. – Thomas K Nov 29 '10 at 22:17
@Thomas: for that you'd need to look at the code or speak to the person who wrote it to be sure. In general, assume the likelihoods aren't comparable across different software/packages/functions. – Gavin Simpson Nov 29 '10 at 22:46