auto.arima not giving the best model according to information criteria

Question

I'm using auto.arima to get the best model for the MASS dataset deaths. However, auto.arima does not seem to give the best model by measures of AIC, AICc, or BIC. auto.arima code below:

> data(deaths, package='MASS')
> deaths
      Jan  Feb  Mar  Apr  May  Jun  Jul  Aug  Sep  Oct  Nov  Dec
1974 3035 2552 2704 2554 2014 1655 1721 1524 1596 2074 2199 2512
1975 2933 2889 2938 2497 1870 1726 1607 1545 1396 1787 2076 2837
1976 2787 3891 3179 2011 1636 1580 1489 1300 1356 1653 2013 2823
1977 3102 2294 2385 2444 1748 1554 1498 1361 1346 1564 1640 2293
1978 2815 3137 2679 1969 1870 1633 1529 1366 1357 1570 1535 2491
1979 3084 2605 2573 2143 1693 1504 1461 1354 1333 1492 1781 1915

> library(forecast)
> auto.arima(deaths)
Series: deaths 
ARIMA(1,0,0)(2,0,0)[12] with non-zero mean 

Coefficients:
         ar1    sar1    sar2  intercept
      0.4418  0.3098  0.5078  2058.2234
s.e.  0.1345  0.0973  0.0998   175.8665

sigma^2 estimated as 79455:  log likelihood=-515.07
AIC=1040.13   AICc=1041.04   BIC=1051.51

If I instead fit a ARIMA(2,1,1)x(1,1,1) model, it is better by all measures:

> Arima(deaths, order=c(2,1,1), seasonal=c(1,1,1))
Series: deaths 
ARIMA(2,1,1)(1,1,1)[12]                    

Coefficients:
         ar1      ar2      ma1     sar1     sma1
      0.2729  -0.3270  -1.0000  -0.2985  -0.9999
s.e.  0.1356   0.1396   0.1305   0.1426   1.0106

sigma^2 estimated as 39753:  log likelihood=-413.08
AIC=838.17   AICc=839.79   BIC=850.64

Answer 1

It is not true that auto.arima is not giving you the best model in terms of information criteria (at least in your example). auto.arima did indeed select a model with higher criteria values than those generated by some other model -- but these criteria values are not comparable. Information criteria cannot be directly compared for models that have different dependent variables from the perspective of calculating the model's likelihood (see Rob J. Hyndman's blog post "Facts and fallacies of the AIC", point 4). In your example, the first model's dependent variable (from the perspective of the model's likelihood) is levels of the original variable while the second model's dependent variable (from the same perspective) is first differences of the original variable; hence, the information criteria are not directly comparable. (Another example could be modelling the variable in levels against modelling its logarithm.)

Also note that auto.arima first determines whether regular or seasonal differencing is needed by using tests (KPSS or alternatively ADF or PP for regular differencing and OCSB or alternatively Canova-Hansen fo seasonal differencing). Given that, it selects optimal AR, MA, SAR and SMA orders based on the information criterion of choice (AICc or alternatively AIC or BIC) using a local search procedure.

Answer 2

Period 38 is a clear outlier. auto.arima isn't looking for outliers. Perhaps use tsoutliers package? Read up on Tsay's work

Here is the model from Autobox. AR2 and AR12 with two outliers. One at period 38 and one at 50. Note that the seasonal differencing is also applied to the outliers.

Adjusted variance 36960.8  

[(1-B**12)]Y(T) = -131.29                               
+[X1(T)][(1-B**12)][(-  249.03    )]        :PULSE             4/  2    38
+[X2(T)][(1-B**12)][(-  452.25    )]        :PULSE             5/  2    50
+     [(1-  .318B** 1+  .373B** 2)(1+  .840B** 12)]**-1  [A(T)]