23
$\begingroup$

I am looking for examples of how to interpret AIC (Akaike information criterion) and BIC (Bayesian information criterion) estimates.

Can negative difference between BICs be interpreted as the posterior odds of one model over the other? How can I put this into words? For example the BIC = -2 may imply that the odds of the better model over the other model are approximately $e^2= 7.4$?

Any basic advice is appreciated by this neophyte.

Improve this question
$\endgroup$
1
  • $\begingroup$ Take a look at the chapter 2. Section 2.6 - which is partially available on books google - might in particular help you. books.google.se/… (Ref: Model Selection and Multi-Model Inference by Kenneth P. Burnham and David R. Anderson. Springer Verlag) $\endgroup$
    – boscovich
    Commented Sep 21, 2012 at 14:03

3 Answers 3

Reset to default
7
$\begingroup$

$AIC$ for model $i$ of an a priori model set can be recaled to $\mathsf{\Delta}_i=AIC_i-minAIC$ where the best model of the model set will have $\mathsf{\Delta}=0$. We can use the $\mathsf{\Delta}_i$ values to estimate strength of evidence ($w_i$) for the all models in the model set where: $$ w_i = \frac{e^{(-0.5\mathsf{\Delta}_i)}}{\sum_{r=1}^Re^{(-0.5\mathsf{\Delta}_i)}}. $$ This is often refered to as the "weight of evidence" for model $i$ given the a priori model set. As $\mathsf{\Delta}_i$ increases, $w_i$ decreases suggesting model $i$ is less plausible. These $w_i$ values can be interpreted as the probability that model $i$ is the best model given the a priori model set. We could also calculate the relative likelihood of model $i$ versus model $j$ as $w_i/w_j$. For example, if $w_i = 0.8$ and $w_j = 0.1$ then we could say model $i$ is 8 times more likely than model $j$.

Note, $w_1/w_2 = e^{0.5\Delta_2}$ when model 1 is the best model (smallest $AIC$). Burnham and Anderson (2002) term this as the evidence ratio. This table shows how the evidence ratio changes with respect to the best model.

Information Loss (Delta)    Evidence Ratio
0                           1.0
2                           2.7
4                           7.4
8                           54.6
10                          148.4
12                          403.4
15                          1808.0

Reference

Burnham, K. P., and D. R. Anderson. 2002. Model selection and multimodel inference: a practical information-theoretic approach. Second edition. Springer, New York, USA.

Anderson, D. R. 2008. Model based inference in the life sciences: a primer on evidence. Springer, New York, USA.

Improve this answer
$\endgroup$
2
  • $\begingroup$ would you mind clarifying what $r$ refers to, in particular since the index of the sum does not appear inside the summation. Does $R$ span the model space? $\endgroup$
    – dopexxx
    Commented Oct 16, 2018 at 18:33
  • $\begingroup$ There are R models in the model set. $\endgroup$
    – RioRaider
    Commented Oct 17, 2018 at 19:43
3
$\begingroup$

I don't think there is any simple interpretation of AIC or BIC like that. They are both quantities that take the log likelihood and apply a penalty to it for the number of parameters being estimated. The specific penalties are explained for AIC by Akaike in his papers starting in 1974. BIC was selected by Gideon Schwarz in his 1978 paper and is motivated by a Bayesian argument.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ The penalty can be interpreted as a prior favoring models of a particular size, though. If you happen to adopt that prior (which has some information-theoretic justifications), then you can calculate a posterior odds ratio directly from IC values. Also, @RioRaider mentions Akaike weights, which give you the probability that a given model is the best model from the set in terms of K-L divergence. (ref--see p. 800). $\endgroup$
    – David J. Harris
    Commented Nov 25, 2012 at 5:39
1
$\begingroup$

You probably use the BIC as a result of approximation to Bayes factor. Therefore you don't consider (more or less) a prior distribution. BIC in a model selection stage is useful when you compare the models. To fully understand BIC, Bayes factor I highly recommend reading an article (sec. 4): http://www.stat.washington.edu/raftery/Research/PDF/socmeth1995.pdf to supplement knowledge with: http://www.stat.washington.edu/raftery/Research/PDF/kass1995.pdf

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.