3
$\begingroup$

I'm using AIC for model selection, and would like to use a relative likelihood measure to quantify how many times a model with minimum AIC (AICmin) fits better than the alternative (with AICi).

For that, I'm using Burnham et al. (2011) formula, which is:

RL = exp ( 0.5 * ( AICmin - AICi ))

The expression is quite easy. However, I'm worried to miss something. In mi case, AICmin is negative (AICmin = -239.10, AICi = 210.43), which makes the difference term (AICmin - AICi) also negative, and thus a relative likelihood on the order of zero (RL = 2.43e-98) and does not make sense.

In the original article I don't find any reference saying that the difference should be absolute, but if so, the ratio becomes too high (RL = 4.11+97) to me to feel sure. Am I missing something? Thank you!

Improve this question
$\endgroup$
2
  • $\begingroup$ The relative likelihood of the best model (the best model of those you are considering), AICmin, is 1. The relative error of the $i$th model, AICi, is ~0 because this model is rather bad compared with the best model. Plot the relative likelihood for an (imaginary) sequence of models with AIC between AICi and AICmin. This might help you get some intuition. $\endgroup$
    – dipetkov
    Feb 4, 2023 at 18:38
  • $\begingroup$ For example: relative_likelihood <- function(x) exp(0.5 * (AICmin - x)); curve(relative_likelihood, from = 210, to = -239, n = 1001) $\endgroup$
    – dipetkov
    Feb 4, 2023 at 18:38

2 Answers 2

Reset to default
2
$\begingroup$

The issue is that this term "relative likelihood" isn't a likelihood at all. The AIC is both a negative log-transformed likelihood (so that lower is better) as well as scaled by a constant related to the number of parameters.

The formula for the AIC is:

$$AIC = 2K - 2 \log(L)$$

It's not surprising to see a log likelihood that is negatively valued, or an AIC that is negatively valued.

In spite of all of that, it's still a separate issue that the relative "likelihood" is close to 0. All the RL is doing is transforming an AIC back to a quasi likelihood ratio. A ratio that's close to 0 means the denominator AICi is very, very big compared to the numerator AICmin. That's not surprising. All it says is AICi really sucks compared to AICmin.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ The real question is why RL is so small when the difference between the two AIC measures is that big (~450). In fact, according to the formula in the original paper, RL decreases as this difference increases, for any sign on AIC. I guess this is a mistake, but I havent been able to find an erratum or a more elaborated explanation on how the formula is deduced $\endgroup$
    – elcortegano
    Dec 19, 2018 at 9:26
  • $\begingroup$ @elcortegano big or small are relative measures. 450 means nothing in isolation. $\endgroup$
    – Firebug
    Dec 16, 2020 at 11:18
  • $\begingroup$ @elcortegano have you tried asking the authors or have any new insight on that? Would be intersting to me as well... $\endgroup$
    – Dunen
    Apr 29, 2022 at 16:01
1
$\begingroup$

Have you considered AIC weights? They serve to normalize the relative likelihoods from a set of candidate models. Also, AIC weights allow quantification of the relative probability that a model is correct (relative to the other models considered) for the given data. See Burnham and Anderson (2002) for more info.

Burnham, K.P., and D.R. Anderson. 2002. Model selection and multimodel inference: a practical information-theoretic approach, 2nd edition. Springer-Verlag, New York, NY. 488 p.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I just come to read about this ans sounds promising. As far as I understand, it consists on dividing this RL for a model i by the sum of all RLs, right? Do you have any reference highlighting why to use it? $\endgroup$
    – elcortegano
    Dec 19, 2018 at 9:31
  • $\begingroup$ In any case, I cannot accept the answer by now, as the original problem prevails. How is it possible that RL decreases as the difference between AIC terms increases, its non sense. $\endgroup$
    – elcortegano
    Dec 19, 2018 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.