1
$\begingroup$

I am currently fitting a mixed model where I analye longitudinal trends in migration between country pairs (68335 observations nested in 6442 groups). One of the first questions I wanted to have answered was whether I should estimate time as linear or exponential. Based on theory and heuristic examination of the plots I have good reason to believe that it should be exponential, but I also wanted to compare the AIC of the linear and exponential models to see which fit the data better. When I tried this, however, I found the difference in AICs to be absurdly high (>480,000). I immediately became suspicious of this but as far as I know, I didn't make any mistakes.

This is the r syntax I used for the models:

modela <- lmer(Migration~(1+Time|Country_number) + Time,REML=FALSE)
modelc <- lmer(log(Migration)~(1+Time|Country_number) + Time,REML=FALSE)

You can see that the only difference is that in model c the dependent variable has been transformed using the natural logarithm.

I then calculated the AICs of both models using the steps outlined HERE

With the following r script:

AIC(modela)

[1] 1050033

AIC(modelc)+2*sum(log(data$Migration),na.rm=TRUE)

[1] 566092.1

You can see that the difference in AICs is $1,050,033 - 566,092 = 483,940.9$. I have heard it say that a difference that is >10 generally means you have a better model fit. However, this large of a difference seems rather excessive.

Did I mistake somewhere, or is this is a normal occurrence?

Thanks in advance.

$\endgroup$
  • 1
    $\begingroup$ @MichaelM - you might want to expand that to an answer, since it is... $\endgroup$ – jbowman Jan 14 at 20:15
2
$\begingroup$

You cannot use AIC to directly compare models with different response.

Why? AIC is a function of the likelihood as well as of the model complexity. The likelihood changes with the response scale and also with the chosen distributional assumption.

So even if the two models are equally complex in terms of number of parameters, the AIC values will completely different since one response is Migration and the other log(Migration).

$\endgroup$
  • $\begingroup$ Thank you for your quick reply. I was not aware of this but that explains the result I got. Is there a way for me to compare the goodness-of-fit between these two models? $\endgroup$ – Twim Jan 14 at 21:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.