In time series models, like ARMA-GARCH, to select appropriate lag or order of the model different information criterion, like AIC, BIC, SIC etc, are used.

My question is very simple, why donot we use adjusted $R^2$ to choose appropriate model? We can select model which lead to higher value of adjusted $R^2$. Because both adjusted $R^2$ and information criterion penalize for additional number of regressors in the model, where former penalize $R^2$ and later penalize likelihood value.

  • 1
    $\begingroup$ I may be missing something in the answers (below) but R-squares as well as Adjusted R-squares are appropriate for the relatively limited class of OLS estimated models whereas AICs, BICs, etc., are appropriate for the broader class of generalized linear models estimated, perhaps, with ML or a variant. $\endgroup$ Feb 18, 2016 at 13:59

2 Answers 2


I would argue that at least when discussing linear models (like AR models), adjusted $R^2$ and AIC are not that different. (This discussion is based on Hansen's Econometrics textbook, if I remember correctly.)

Consider the question of whether $X_2$ should be included in $$ y=\underset{(n\times K_1)}{X_1}\beta_1+\underset{(n\times K_2)}{X_2}\beta_2+\epsilon $$ This is equivalent to comparing the models \begin{eqnarray*} \mathcal{M}_1&:&y=X_1\beta_1+u\\ \mathcal{M}_2&:&y=X_1\beta_1+X_2\beta_2+u, \end{eqnarray*} where $E(u|X_1,X_2)=0$. We say that $\mathcal{M}_2$ is the true model if $\beta_2\neq0$. Notice that $\mathcal{M}_1\subset\mathcal{M}_2$. The models are thus nested. A model selection procedure $\widehat{\mathcal{M}}$ is a data-dependent rule that selects the most plausible of several models.

We say $\widehat{\mathcal{M}}$ is consistent if \begin{eqnarray*} \lim_{n\rightarrow\infty}P\bigl(\widehat{\mathcal{M}}=\mathcal{M}_1|\mathcal{M}_1\bigr)&=&1\\ \lim_{n\rightarrow\infty}P\bigl(\widehat{\mathcal{M}}=\mathcal{M}_2|\mathcal{M}_2\bigr)&=&1 \end{eqnarray*}

Consider adjusted $R^2$. That is, choose $\mathcal{M}_1$ if $\bar{R}^2_1>\bar{R}^2_2$. As $\bar{R}^2$ is monotonically decreasing in $s^2$, this procedure is equivalent to minimizing $s^2$. In turn, this is equivalent to minimizing $\log(s^2)$. For sufficiently large $n$, the latter can be written as \begin{eqnarray*} \log(s^2)&=&\log\left(\widehat{\sigma}^2\frac{n}{n-K}\right) \\ &=&\log(\widehat{\sigma}^2)+\log\left(1+\frac{K}{n-K}\right) \\ &\approx&\log(\widehat{\sigma}^2)+\frac{K}{n-K} \\ &\approx&\log(\widehat{\sigma}^2)+\frac{K}{n}, \end{eqnarray*} where $\widehat{\sigma}^2$ is the ML estimator of the error variance. Model selection based on $\bar{R}^2$ is therefore asymptotically equivalent to choosing the model with the smallest $\log(\widehat{\sigma}^2)+K/n$. This procedure is inconsistent.

Proposition: $$\lim_{n\rightarrow\infty}P\bigl(\bar{R}^2_1>\bar{R}^2_2|\mathcal{M}_1\bigr)<1$$

Proof: \begin{eqnarray*} P\bigl(\bar{R}^2_1>\bar{R}^2_2|\mathcal{M}_1\bigr)&\approx&P\bigl(\log(s^2_1)<\log(s^2_2)|\mathcal{M}_1\bigr) \\ &=&P\bigl(n\log(s^2_1)<n\log(s^2_2)|\mathcal{M}_1\bigr) \\ &\approx&P(n\log(\widehat{\sigma}^2_1)+K_1<n\log(\widehat{\sigma}^2_2)+K_1+K_2|\mathcal{M}_1) \\ &=&P(n[\log(\widehat{\sigma}^2_1)-\log(\widehat{\sigma}^2_2)]<K_2|\mathcal{M}_1) \\ &\rightarrow&P(\chi^2_{K_2}<K_2) \\ &<&1, \end{eqnarray*} where the 2nd-to-last line follows because the statistic is the LR statistic in the linear regression case that follows an asymptotic $\chi^2_{K_2}$ null distribution. QED

Now consider Akaike's criterion, $$ AIC=\log(\widehat{\sigma}^2)+2\frac{K}{n} $$ Thus, the AIC also trades off the reduction of the SSR implied by additional regressors against the "penalty term," which points in the opposite direction. Thus, choose $\mathcal{M}_1$ if $AIC_1<AIC_2$, else select $\mathcal{M}_2$.

It can be seen that the $AIC$ is also inconsistent by continuing the above proof in line three with $P(n\log(\widehat{\sigma}^2_1)+2K_1<n\log(\widehat{\sigma}^2_2)+2(K_1+K_2)|\mathcal{M}_1)$. The adjusted $R^2$ and the $AIC$ thus choose the "large" model $\mathcal{M}_2$ with positive probability, even if $\mathcal{M}_1$ is the true model.

As the penalty for complexity in AIC is a little larger than for adjusted $R^2$, it may be less prone to overselect, though. And it has other nice properties (minimizing the KL divergence to the true model if that is not in the set of models considered) that are not addressed in my post.

  • 2
    $\begingroup$ Great answer: not too heavy but still exact! If it had been there yesterday, I would have not posted mine. $\endgroup$ Feb 18, 2016 at 14:37
  • $\begingroup$ What about for the ARMA-GARCH case? How would $R^2_{adj} $ do at selecting amung MA and GARCH terms? $\endgroup$ Feb 18, 2016 at 17:46
  • $\begingroup$ I wouldn't dare to say. As you explain it's not even clear what R2 means for the fit of such a model. $\endgroup$ Feb 18, 2016 at 18:31

The penalty in $R^2_{adj}$ does not yield the nice properties in terms of model selection as posessed by the AIC or BIC. The penalty in $R^2_{adj}$ is enough to make $R^2_{adj}$ an unbiased estimator of the population $R^2$ when none of the regressors actually belongs to the model (as per Dave Giles' blog posts "In What Sense is the "Adjusted" R-Squared Unbiased?" and "More on the Properties of the "Adjusted" Coefficient of Determination"); however, $R^2_{adj}$ is not an optimal model selector.

(There could be a proof by contradiction: if AIC is optimal in one sense and BIC is optimal in another, and $R^2_{adj}$ is not equivalent to either of them, then $R^2_{adj}$ is not optimal in either of these two senses.)

  • $\begingroup$ How many GARCH parameters do I have to add before the $R^2$ increases? :)....I believe a similar argument could be made for the assumption of correlated errors (as in an MA model), A GLS model does not reduce the sum of squared residuals over ordinary least squares. In both MA and GARCH, parameters (not explanatory variables, which $R^2{adj}$ is adjusted for) are added to the model. MA and GARCH parameters are not added to reduce $SSR$, rather they re added to increase the likelihood and/or decreases a weighted sum of squared residuals to reflect the lack of iid error terms. $\endgroup$ Feb 17, 2016 at 21:07
  • $\begingroup$ Does this actually address the original post or my answer? In any case, I agree with your points. $\endgroup$ Feb 17, 2016 at 21:15
  • $\begingroup$ What I was trying to point out is that $R^2_{adj}$ can't really be used to select upon GARCH components (and possibly MA components too) since it is based on the fraction of $SST-SSR$ over $SST$ which are biased estimators of variance when the error terms are not iid. (this is just a specific case of the bias you where talking about). In the case of ARMA-GARCH, you would never select a model with GARCH components, even if there was stochastic volatility in the data, because it does not increase $R^2$. Basically, I am agreeing with you by trying to give specific examples. $\endgroup$ Feb 17, 2016 at 21:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.