AIC model selection when successive models have ΔAIC <2 compared to next best model

Question

I have a set of linear mixed effects models with which I'm changing fixed effects, and comparing AIC for model selection. Something akin to this hypothetical model set. Sample is ~750 observations so not using AICc.

M0 <- lmer( y ~ x + (1|p) + (1|q/r), data = my.data, REML = FALSE ) 
M1 <- lmer( y ~ x + y + (1|p) + (1|q/r), data = my.data, REML = FALSE )
M2 <- lmer( y ~ x + z + (1|p) + (1|q/r),  data = my.data, REML = FALSE )
M3 <- lmer( y ~ x + y + z + (1|p) + (1|q/r), data = my.data, REML = FALSE )

When I check the AIC for model selection, the models progressively become a better fit with the data set, with AIC values progressively decreasing in the model set, similar to this.

AIC(MO)
[1] 166.3
AIC(M1)
[1] 165.1
AIC(M2)
[1] 163.4
AIC(M3)
[1] 161.8

According to the general rule applied, two models with ΔAIC <2 are statistically indistinguishable and there is not enough evidence to select a single model over the other.

However, in a situation such as this, where there is ΔAIC >2 between best and worst fitting models, but there is ΔAIC <2 between each model and the next best fitting model, what would be the correct interpretation of AIC values? What would be the appropriate model selection, if any?

Answer 1

One way of thinking about this is to consider model averaging with AIC based weights proportional to $\exp(-AIC/2)$, which would be normalized to sum to 1.

That illustrates that even at AIC difference of 2 the seemingly less well fitting model still gets a non-negligible weight. Thus, selecting a single model is very problematic due to the considerable model uncertainty and should typically not be done. Model averaging is usually a much better approach.