0
$\begingroup$

I have a set of linear mixed effects models with which I'm changing fixed effects, and comparing AIC for model selection. Something akin to this hypothetical model set. Sample is ~750 observations so not using AICc.

M0 <- lmer( y ~ x + (1|p) + (1|q/r), data = my.data, REML = FALSE ) 
M1 <- lmer( y ~ x + y + (1|p) + (1|q/r), data = my.data, REML = FALSE )
M2 <- lmer( y ~ x + z + (1|p) + (1|q/r),  data = my.data, REML = FALSE )
M3 <- lmer( y ~ x + y + z + (1|p) + (1|q/r), data = my.data, REML = FALSE )

When I check the AIC for model selection, the models progressively become a better fit with the data set, with AIC values progressively decreasing in the model set, similar to this.

AIC(MO)
[1] 166.3
AIC(M1)
[1] 165.1
AIC(M2)
[1] 163.4
AIC(M3)
[1] 161.8

According to the general rule applied, two models with ΔAIC <2 are statistically indistinguishable and there is not enough evidence to select a single model over the other.

However, in a situation such as this, where there is ΔAIC >2 between best and worst fitting models, but there is ΔAIC <2 between each model and the next best fitting model, what would be the correct interpretation of AIC values? What would be the appropriate model selection, if any?

$\endgroup$
1
$\begingroup$

One way of thinking about this is to consider model averaging with AIC based weights proportional to $\exp(-AIC/2)$, which would be normalized to sum to 1.

That illustrates that even at AIC difference of 2 the seemingly less well fitting model still gets a non-negligible weight. Thus, selecting a single model is very problematic due to the considerable model uncertainty and should typically not be done. Model averaging is usually a much better approach.

$\endgroup$
  • $\begingroup$ Thanks Björn, I did indeed find the best approach was using MuMIn::dredge() with a slightly different global model, followed by model averaging in the end to infer conclusions. $\endgroup$ – Roasty247 Feb 19 '18 at 8:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.