P-values shows something different than AIC or delta AIC. What to do?

Question

I applied an glmer model on data derived from several raster (image) files. Together, my dataset had several hundred thousand rows.

My model was something like this:

glmer(subject_0_1 ~ var1 + (1 | Year), data = datsc, family = binomial(link = "logit"))

I had three highly correlated variables, so I 'v used three models:

glmer(subject_0_1 ~ var2 + (1 | Year), data = datsc, family = binomial(link = "logit"))

glmer(subject_0_1 ~ var3 + (1 | Year), data = datsc, family = binomial(link = "logit"))

Each model told me, that my explanatory variables is statistically significant in relation to subject_0_1.

At the end, I compared my models with AIC, the one with the var3 had the lowest AIC value. But when I calculated the deltaAIC, it showed that the first two models are far from valid - their values were more than deltaAIC > 100. So, variables var1 and var2 are importatnt from the viewpoint of p-values, but deltaAIC shows that they are meaningles?

Answer 1

Yes!

AIC is based on the log-likelihood and penalize according the number of parameters. So, AIC aim to find the model that maximize the likelihood of the model and the data (simplifying).

P-values, in this case, are related to this variable effect and their errors.

A variable can have a significant effect, but do not necessarily it is the best predictor in the likelihood point of view.

And remember, AIC is to compare models, suppose you do not have var3 and only have var1 and var2, if you compare the models, the AIC will rank the models with var1 and var2 and you will pick the best(s) model(s) only with one of these variables.

Hence, you are comparing very different things.

In this case, you pick the model with deltaAIC<2 or 4 (see Burnham et al. 2010). Since in you case it is only one model, you can assume that the variable which best describes your data is var3 and put var1 and var2 out according AIC.

If your best model have a non-significant variable, probably the null model is the best to describe your data, so, you can assume that your variables are not parsimonious to explain your data. (It is not your case)

I also suggest to use AICc, which is indicated to small sample sizes, if you have a big sample, the AICc will converge to AIC, thus, AICc is a safety value.

Best