4
$\begingroup$

Can I compare AIC values of a linear function with a non-linear function? Because I get totally different results. One is 4000 other the 6000000. Estimation is done on the same data setvariables.

$\endgroup$
  • 1
    $\begingroup$ Can you please elaborate? $\endgroup$ – user88 Sep 27 '13 at 10:59
4
$\begingroup$

Looks like one of them doesn't really fit the data.

As long as you did not transform the response variable -- for example, replacing $y$ by $\log(y)$ -- you can use very different models, for example you can compare $y = b \cdot x$ with $y = e^{b \cdot x}$ or $y = a\cdot x^b$.

However, you are not allowed to use AIC for comparing $y = b\cdot x$ with $\log(y) = b \cdot x$.

$\endgroup$
4
$\begingroup$

Well, they seem to be fitted with different algorithms and the likelihoods (or AIC) are calculated with different methods. Please check about it with your software. Different software usually uses different methods to calculate the likelihood and often adds a constant to the likelihood or AIC for convenience. That means the scales are totally different.

Regarding to AIC comparison with different class of models, there are some arguing whether they can be compared or not. Prof Ripley (2002, 2004 http://www.stats.ox.ac.uk/~ripley/Nelder80.pdf, and some other posts on forum) said AIC must be compared for nested model, while Anderson and Burnham (2006 https://sites.warnercnr.colostate.edu/anderson/wp-content/uploads/sites/26/2016/11/AIC-Myths-and-Misunderstandings.pdf) claimed it is not. Moreover Ripley also said that AIC can only be used for MLE method, while the non-linear model is fitted by nonlinear (weighted) least-square rather than MLE.

In my opinion, AIC can be used for some non-nested models like Y~A+B and Y~A+C, but at least the same class of model. Supposing that the (+2*p) of AIC as the complexity penalty, it seems that linear and non-linear models, and even spline, with the same number of parameters are not the same complex.

$\endgroup$
2
$\begingroup$

From the description you have given, yes. This is exactly the case where you would want to use AIC and the like, differing models to the same data. The model with the higher value is the worse model. And if the $\Delta \mbox{AIC} >10$ there is a hardly any evidence for the worse model.

However, to make sure there is no error, I would check the model's fits and see if the one really does fit that bad.

$\endgroup$
  • $\begingroup$ What about the case where linear and the non-linear functions are fitted with different algorithms, so that the resulting likehoods (and therefore AICs) of the models are not directly comparable? $\endgroup$ – JTT Sep 27 '13 at 12:31
  • 1
    $\begingroup$ The algorithm is not the critical point. The values you base your AIC on e.g., deviance, likelihood, or $G^2$, obviously need to be the same. And this is what I perhaps meant with my last sentence. $\endgroup$ – Henrik Sep 27 '13 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.