5
$\begingroup$

The wikipeda-page for AIC states: "it (AIC) deals with the trade-off between the goodness of fit of the model and the complexity of the model". If I understand this correctly, this means that AIC is not a measure of goodness of fit, but merely includes a term (the maximum likelihood) which evaluates goodness of fit.

The wikipedia-page for goodness-of-fit, however, lists AIC as a measure for "assessing whether a given distribution is suited to a data-set". I interpret this that AIC is considered a measure of goodness of fit.

So, is AIC a measure of goodness of fit or not?

This question is not about how AIC is defined or what it is, but if it fits into the definition "a measure of goodness of fit". Therefore, it is not a duplicate question.

Improve this question
$\endgroup$
2
  • $\begingroup$ AIC helps to select models by penalizing the likelihood function. $\endgroup$
    – Michael R. Chernick
    Commented Apr 1, 2017 at 16:35
  • 2
    $\begingroup$ Scroll down to the answer of jwimberley here for a useful interpretation. $\endgroup$
    – Richard Hardy
    Commented Apr 3, 2017 at 8:26

2 Answers 2

Reset to default
9
$\begingroup$

Just to expand a little on Hossein's answer: AIC is a measure of relative goodness of fit. If you take a model and calculate its AIC then you might get a value of, say, 2000. That number on its own is meaningless, and tells you nothing about how well your model fits. However, say you then fit another model which contains one more explanatory variable. When you calculate the AIC again, you see that it is dropped to 1500. That is now evidence that model 2 is a better fit to the data than model 1.

AIC is useful for comparing models, but it does not tell you anything about the goodness of fit of a single, isolated model.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Elaboration: it is a measure of relative goodness of fit for a specific set of data. You can't compare the fit of model A on data A vs model B on data B using AIC $\endgroup$
    – russellpierce
    Commented Apr 3, 2017 at 4:45
  • 1
    $\begingroup$ So to make it clear, is the answer to the OP's question yes or no? Also, scroll down to the answer of jwimberley here for a hint why the value of AIC is not quite meaningless, as it indicates the expected out-of-sample error rate. $\endgroup$
    – Richard Hardy
    Commented Apr 3, 2017 at 8:27
4
$\begingroup$

AIC like many other model quality measures has two parts: goodness of fit and model simplicity. If you only measure the quality of a model by its goodness of fit, it favors overfitted models. On the other hand, if you only measure the model quality by its simplicity, it favors underfitted models. Therefore, AIC considers both criteria in evaluating a model.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ You do not seem to answer the question. It is nice to give an explanation of AIC (even though there are quite a few threads already doing that), but you should try to answer the question. $\endgroup$
    – Richard Hardy
    Commented Apr 3, 2017 at 8:28
  • $\begingroup$ I think I have answered the question. Julian asked wheter AIC is a measure of goodness of fit or not. I tried to show that it measures not only goodness of fit but also the model complexity. $\endgroup$
    – Hossein
    Commented Apr 3, 2017 at 9:40
  • $\begingroup$ Exactly, and you did that without answering the question. My point was, the answer to a question "Is [subject] [object]?" is "Yes" or "No" or "I don't know", rather than "[Subject] has property A and property B". $\endgroup$
    – Richard Hardy
    Commented Apr 3, 2017 at 9:48
  • $\begingroup$ The answer to this question is neither "yes" nor "no". The correct answer is "AIC does not only measure the goodness of fit but also the model complexity" as mentioned in my answer. $\endgroup$
    – Hossein
    Commented Apr 3, 2017 at 9:55
  • $\begingroup$ Thanks for the clarification. I would suggest the answer is "No", based on what I learned from your answer and comments. $\endgroup$
    – Richard Hardy
    Commented Apr 3, 2017 at 10:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.