# Overfitting when using corrected AIC for model selection

I am using the corrected AIC to select the lag order in a simple AR(p) model. I chose the the AICc since my sample is fairly small (n=135). The AICc minimal model is the AR(15). To me it seems like an overfit to include 15 lags in such a small sample.

Do you agree? And does anyone know a rule of thumb for max lag order relative to sample size?

From the arfunction in the stats package in R:

order.max
Maximum order (or order) of model to fit. Defaults to the smaller of N-1 and 10*log10(N) where N is the number of observations except for method="mle" where it is the minimum of this quantity and 12.

To me these default make little sense. Where do they come from?

• I have heard that decreasing between 3 and 5 degrees of freedom can help, but I also have dealt with spline methods where I can have effective non-integer parameter values, and so in that case it gets messier. May 20 '13 at 21:58
• Care to be more specific about the data? If so, what is the series being modeled? Have you tried inspecting the acf and pacf? You might find a more parsimonious model if you try doing that. May 20 '13 at 22:34
• Also, by using the ar() function, are you deciding a priori that an AR model provides an adequate representation of the data generating mechanism? In other words, does the ar() function rule out the possibility of adding MA, seasonal AR, seasonal MA, and "difference" terms to the model? May 20 '13 at 22:43
• I am using it to forecast monthly house prices. I have inspected the acf and pacf, but they dont give much information since the AR(p) does not seem to fit the data well. That is, yes I'm deciding apriori that the model is a good fit. The reason for this: I have a set of models (ARIMA and exponential smoothing models). The AR(p) is only included as a benchmark in the set, as is quite common in the forecasting literature. May 20 '13 at 22:48
• AICc is substantially more conservative than the uncorrected AIC in small samples. If it's still choosing too large a model for your purposes, you might be better served by BIC, which is more conservative than AIC -- but on sufficiently small samples there's no guarantee it won't choose a larger model than AICc. May 21 '13 at 1:23

You need to use the likelihood for the whole sample from the first principles based on $$\log L \sim -\frac12 ({\bf y}-\mathbf{\mu})'\Sigma(\theta)^{-1}({\bf y}-\mathbf{\mu}) - \frac12 \log |\Sigma(\theta)|$$ where ${\bf y}\in \mathbf{R}^{135}$ is your whole sample vector, and $\Sigma(\theta)$ is the model-implied covariance matrix of your ARMA($p,q$) process. God only knows what your naive AIC calculation for i.i.d. data is actually doing; it is out of context and has little value here.
AIC's for selecting the "best" model from a candidate set*, so consideration of overfit has to be relative to the other models included in that set. I'd wager the AR(15) model has substantial overfit compared to other models not included in the candidate set, such as those suggested by @Graeme (an MA(1) model is equivalent to an AR($\infty$) model), but if for some reason you're not interested in those, there's no reason not to stick with it as the best of a bad lot.