Using AIC, for model selection when both models are equally weighted, and one model has fewer parameters

Question

I am using AIC (Akaike information criterion) for model selection. There are 2 models. The first model has 2 parameters with log likelihood of -10182.0284 and the second model has 3 parameters with the same likelihood when tried on a specific dataset that displays the need for only two parameters. The weighting I get with AIC is equal for both models. The equality seems to come from the fact that not all significant figures are taken into account and with such low log likelihood the number of parameter penalization is insignificant. The results:

AICmodelSelect(-10182.0284,-10182.0284)
AIC_min
null model min AIC
relprob_null
     1
relprob_alt
     1
weight_null
    0.5000
weight_alt
    0.5000

AIC equally favours both models. I am also doing likelihood ratio test cause the models are nested and the p-value is below 0.01 for the null model (simpler constrained model). But how do I justify choosing the simpler model with AIC when there is equal weighting given here?

Answer 1

There was a fairly good commentary in the Journal of Wildlife Management concerning uninformative parameters within the AIC framework.

Arnold, T. W. 2010. Uninformative parameters and model selection using Akaike’s Information Criterion. Journal of Wildlife Management 74:1175–1178. [Link].

We usually consider models within 2 delta AIC as competitive. However, if a model has an addition of only one parameter to its competitor and that parameter is not significant, that parameter is likely spurious. AIC = –2LL + 2K so the penalty for adding one parameter is +2 AIC. If only one parameter is added but the AIC is within 2 delta AIC, the model fit was not improved enough to overcome the penalty. Therefore, that parameter is uninformative and should not be included in the model or interpreted as having an effect.

Answer 2

I couldn't find AICmodelSelect in any R package, searching in both R with ?? and Google. What package did you use? Or is it R?

In any case, if the log likelihoods are equal and the models have different numbers of parameters, then the AIC are not equal, which is what you have entered. The formula for AIC is $AIC = 2k - 2ln(L)$ where k is the number of parameters and $2ln(L)$ is the log likelihood.

In your case the two AICs would be 6 + 10182.0284 and 4 + 10182.0284, the second is smaller and that is the model you should choose, based on AIC.

Answer 3

Why do people strictly rely upon a criteria (ie AIC) to determine the "best" model? Why not use the principle of N.I.I.D and parsimony as the guide instead of a fit statistic? Sure we can compare variance to after that to see who had a better model, but this whole rule based way of modeling is contra to what I believe in.

As you may know, N.I.I.D. is what we are first taught in time series analysis that the errors should be gaussian. By using AIC or BIC criterion to build a model you are losing the goal of building a model and more of fitting. I have found that instead of using AIC that focusing on the N.I.I.D. of the errors, significance of parameters and parsimony you will have a better model using an Identification scheme focused on robustified ACF and PACF.