# Difference in AIC between AR(p) and AR(p+1) models

I have a situation where my AR(2) model has a lower AIC value (-98.49) than both the AR(1) (-75.84) and the AR(3) model (-92.91). As you can see, the difference in AIC between the AR(2) and the AR(3) is 5.58.

However, as I understand it, from the definition of the AIC, -2*LogLik+2*npar, it follows that the absolute difference in AIC between an AR(p) and AR(p+1) model can never be >2, since this would imply that the log-likelihood would have decreased after adding a parameter.

I have checked to confirm; the log-likelihood as computed by the TSA package (my_AR_model$loglik) is in fact higher for the AR(2), (52.25), than for the AR(3), (50.45). I have also confirmed that R isn't calculating the AICc, since this would only add 1.2 to the AIC (I have 24 observations). So my question is: am I missing something? Can the log-likelihood actually decrease when adding a parameter, is there something special about how the TSA package calculates AIC, am I doing something wrong in R, or is R pulling my leg somehow? EDIT: The code and data I have been using are the following: Data: my_vector <- c(-0.15117448, -0.14348934, -0.18137095, -0.19605340, -0.20543727, -0.21709754, -0.21490577, -0.20853185, -0.19525812, -0.14138660, -0.12247660, -0.07981194, -0.01453317, 0.05647378, 0.11952508, 0.20328388, 0.26107555, 0.30314216, 0.29448029, 0.28686523, 0.22518196, 0.14278947, 0.07233569, 0.06609976)  Code: my_timeseries <- ts(my_vector, frequency = 1) ar2 <- arima(my_timeseries, order = c(2, 0, 0), method = "ML") ar3 <- arima(my_timeseries, order = c(3, 0, 0), method = "ML") ar2$loglik

ar3$loglik  • can we have a reproducible example please? Can you try with stats::arima and see if you get the same answers as with TSA::arima ? – Ben Bolker Dec 11 '17 at 0:57 • Thank you for the tip, I tried with stats now and it seems to work! – Frankie Dec 11 '17 at 9:43 • If you find a reproducible problem, could you please contact the maintainer of the TSA package (maintainer("TSA")) ? – Ben Bolker Dec 11 '17 at 13:05 ## 2 Answers Can the log-likelihood actually decrease when adding a parameter, is there something special about how the TSA package calculates AIC, am I doing something wrong in R, or is R pulling my leg somehow? Yes, it can decrease when adding a parameter. Adding parameters usually increases the log-likelihood, and may substantially increase it, so much so that the decrease in negative twice this might outweigh the increase in the penalty term, which results in a decreased AIC. It does appear difficult to get the AIC statistic to decrease by more than$2$adding a parameter, however. Using some algebra, you can see that $$-2\ell_1 + 2p - (-2 \ell_2 + 2[p+1]) > 2$$ if and only if $$-2(\ell_1-\ell_2) + 2(p - (p+1)) > 2$$ if and only if $$\ell_2 - \ell_1 > 2$$ where$\ell_i$is the log-likelihood for model$i=1,2$, and$p$is the number of parameters for the smaller model. You can see it's unlikely, though, because this happens if and only if the likelihood ratio is bigger than$e^2\$. I can't see why it can't happen at the moment, but I've tried a few examples and it seems to not like to go past it.

fakeData <- arima.sim(n = 200, list(ar = c(1.8, -.9), sd = sqrt(0.01))); plot.ts(fakeData)
AIC(arima(fakeData,order=c(3,0,0))); AIC(arima(fakeData,order=c(2,0,0))); AIC(arima(fakeData,order=c(1,0,0)));
#  606.0292
#  604.5039
#  922.0545

• Thank you for the reply! However, my problem is not that the AIC statistic decreases after adding a parameter, but that it increases by more than two. The AR(2) has an AIC of -98.49. When adding one parameter, so that I have an AR(3) model, the AIC increases to -92.91. This is an increase of 5.58. Which means that the AR(3), i.e. model with three parameters, has a LOWER log-likelihood than the AR(2), i.e. model with only two parameters. – Frankie Dec 10 '17 at 23:28