# P-values shows something different than AIC or delta AIC. What to do?

I applied an glmer model on data derived from several raster (image) files. Together, my dataset had several hundred thousand rows.

My model was something like this:

glmer(subject_0_1 ~ var1 + (1 | Year), data = datsc,


I had three highly correlated variables, so I 'v used three models:

glmer(subject_0_1 ~ var2 + (1 | Year), data = datsc,

glmer(subject_0_1 ~ var3 + (1 | Year), data = datsc,


Each model told me, that my explanatory variables is statistically significant in relation to subject_0_1.

At the end, I compared my models with AIC, the one with the var3 had the lowest AIC value. But when I calculated the deltaAIC, it showed that the first two models are far from valid - their values were more than deltaAIC > 100. So, variables var1 and var2 are important from the viewpoint of p-values, but deltaAIC shows that they are meaningless?

• Why should $p$-values and AIC give the same result? They are based on different (though related) principles and are answering different questions. Perhaps this brief thread could be helpful? Commented Mar 27, 2019 at 14:43

Yes!

AIC is based on the log-likelihood and penalize according the number of parameters. So, AIC aim to find the model that maximize the likelihood of the model and the data (simplifying).

P-values, in this case, are related to this variable effect and their errors.

A variable can have a significant effect, but do not necessarily it is the best predictor in the likelihood point of view.

And remember, AIC is to compare models, suppose you do not have var3 and only have var1 and var2, if you compare the models, the AIC will rank the models with var1 and var2 and you will pick the best(s) model(s) only with one of these variables.

Hence, you are comparing very different things.

In this case, you pick the model with deltaAIC<2 or 4 (see Burnham et al. 2010). Since in you case it is only one model, you can assume that the variable which best describes your data is var3 and put var1 and var2 out according AIC.

If your best model have a non-significant variable, probably the null model is the best to describe your data, so, you can assume that your variables are not parsimonious to explain your data. (It is not your case)

I also suggest to use AICc, which is indicated to small sample sizes, if you have a big sample, the AICc will converge to AIC, thus, AICc is a safety value.

Best

• Regarding your first paragraph and the aim of AIC, I am not sure I find it clear. You could say, though, that AIC aims at finding a model that will have the highest likelihood on a new observation drawn from the same population. The paragraph starting with If your best model is also a little confusing to me. Also, could you give a full reference for Burnham et al. 2011? Commented Mar 27, 2019 at 16:31
• Sorry for this confusion. Yes, the highest likelihood with your population and due the parameter penalizations it aims to have the highest likelihood with unobserved data too. researchgate.net/publication/… Commented Mar 28, 2019 at 14:43