2
$\begingroup$

I have a situation where my AR(2) model has a lower AIC value (-98.49) than both the AR(1) (-75.84) and the AR(3) model (-92.91). As you can see, the difference in AIC between the AR(2) and the AR(3) is 5.58.

However, as I understand it, from the definition of the AIC, -2*LogLik+2*npar, it follows that the absolute difference in AIC between an AR(p) and AR(p+1) model can never be >2, since this would imply that the log-likelihood would have decreased after adding a parameter.

I have checked to confirm; the log-likelihood as computed by the TSA package (my_AR_model$loglik) is in fact higher for the AR(2), (52.25), than for the AR(3), (50.45).

I have also confirmed that R isn't calculating the AICc, since this would only add 1.2 to the AIC (I have 24 observations).

So my question is: am I missing something? Can the log-likelihood actually decrease when adding a parameter, is there something special about how the TSA package calculates AIC, am I doing something wrong in R, or is R pulling my leg somehow?

EDIT: The code and data I have been using are the following:

Data:

my_vector <- c(-0.15117448, -0.14348934, -0.18137095, -0.19605340, -0.20543727, -0.21709754, -0.21490577, -0.20853185, -0.19525812, -0.14138660, -0.12247660, -0.07981194, -0.01453317,  0.05647378,  0.11952508, 0.20328388,  0.26107555,  0.30314216,  0.29448029,  0.28686523,  0.22518196,  0.14278947,  0.07233569,  0.06609976)

Code:

my_timeseries <- ts(my_vector, frequency = 1)

ar2 <- arima(my_timeseries, order = c(2, 0, 0), method = "ML")

ar3 <- arima(my_timeseries, order = c(3, 0, 0), method = "ML")

ar2$loglik

ar3$loglik
$\endgroup$
  • $\begingroup$ can we have a reproducible example please? Can you try with stats::arima and see if you get the same answers as with TSA::arima ? $\endgroup$ – Ben Bolker Dec 11 '17 at 0:57
  • $\begingroup$ Thank you for the tip, I tried with stats now and it seems to work! $\endgroup$ – Frankie Dec 11 '17 at 9:43
  • $\begingroup$ If you find a reproducible problem, could you please contact the maintainer of the TSA package (maintainer("TSA")) ? $\endgroup$ – Ben Bolker Dec 11 '17 at 13:05
3
$\begingroup$

Can the log-likelihood actually decrease when adding a parameter, is there something special about how the TSA package calculates AIC, am I doing something wrong in R, or is R pulling my leg somehow?

Yes, it can decrease when adding a parameter. Adding parameters usually increases the log-likelihood, and may substantially increase it, so much so that the decrease in negative twice this might outweigh the increase in the penalty term, which results in a decreased AIC.

It does appear difficult to get the AIC statistic to decrease by more than $2$ adding a parameter, however. Using some algebra, you can see that $$ -2\ell_1 + 2p - (-2 \ell_2 + 2[p+1]) > 2 $$ if and only if $$ -2(\ell_1-\ell_2) + 2(p - (p+1)) > 2 $$ if and only if $$ \ell_2 - \ell_1 > 2 $$ where $\ell_i$ is the log-likelihood for model $i=1,2$, and $p$ is the number of parameters for the smaller model. You can see it's unlikely, though, because this happens if and only if the likelihood ratio is bigger than $e^2$. I can't see why it can't happen at the moment, but I've tried a few examples and it seems to not like to go past it.

fakeData <- arima.sim(n = 200, list(ar = c(1.8, -.9), sd = sqrt(0.01))); plot.ts(fakeData)
AIC(arima(fakeData,order=c(3,0,0))); AIC(arima(fakeData,order=c(2,0,0))); AIC(arima(fakeData,order=c(1,0,0))); 
# [1] 606.0292
# [1] 604.5039
# [1] 922.0545
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for the reply! However, my problem is not that the AIC statistic decreases after adding a parameter, but that it increases by more than two. The AR(2) has an AIC of -98.49. When adding one parameter, so that I have an AR(3) model, the AIC increases to -92.91. This is an increase of 5.58. Which means that the AR(3), i.e. model with three parameters, has a LOWER log-likelihood than the AR(2), i.e. model with only two parameters. $\endgroup$ – Frankie Dec 10 '17 at 23:28
  • $\begingroup$ @J.Frank I could say more, perhaps, if you provided more information about your data/model/code. $\endgroup$ – Taylor Dec 10 '17 at 23:52
  • $\begingroup$ I have added the data and code in an edit! $\endgroup$ – Frankie Dec 11 '17 at 10:02
1
$\begingroup$

The AIC is like a decoration on a christmas/holiday tree and you should also know that there is no Santa Claus. Additionally model parameters are simultaneously optimized thus they change based upon the model AND the # of linear equations used .I would put less belief on the AIC (unless you have the same # of fitting points) and refer model selection to tests of necessity and sufficiency and ITERATE in a cautious way to that better model.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ What does this mean $\endgroup$ – user217285 Dec 11 '17 at 21:01
  • $\begingroup$ It explains that an AIC premises a particular # of estimating equations. If the y differ then the AIC's are not necessarily comparable. $\endgroup$ – IrishStat Dec 11 '17 at 21:26
  • $\begingroup$ Could you expand on "refer model selection to tests of necessity and sufficiency and ITERATE in a cautious way to that better model"? If necessarily I'll make a new question. $\endgroup$ – user217285 Dec 11 '17 at 21:47
  • $\begingroup$ please make a new question and if you wish submit a data set of your choosing and I will try and be explicit .... $\endgroup$ – IrishStat Dec 11 '17 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.