Can you use AIC/BIC to compare models on untransformed data with models on transformed data (such as log, inverse hyperbolic sine, etc.)? I.e. if a model using logged data gives an AIC = 53.62 and a model using the the non-logged data gives an AIC = 150.6, can you say that the logged model is better even though the scale of the data is different between the two models?
$\begingroup$
$\endgroup$
2
-
1$\begingroup$ No, you can't. I guess this has been answered before, you may look up the existing posts on Cross Validated to find it. $\endgroup$– Richard HardyCommented Jan 15, 2016 at 14:35
-
2$\begingroup$ No not directly. However sometimes it is possible. You need to adjust the likelihood with the appropriate probability transform, incorporating the Jacobian. $\endgroup$– Cagdas OzgencCommented Jan 15, 2016 at 15:54
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Your question appears to be a special case of this post which explicitly considers prerequisites for AIC model comparison. One of the important prerequisites is that the dependent variable (the vector of observations) is exactly the same across the models being compared; see the answer by @usεr11852, especially his point 2. The dependent variable cannot be a transformation and it cannot have different number of observations in different models (which may effectively happen, for example, when comparing ARIMA and exponential smoothing models estimated on exactly the same set of data (!); read carefully the brief answer to this post).
However, transformed dependent variables can in principle be handled by adjusting the AIC for the transformation: see a comment by @probabilityislogic under his/her answer here and the comment by @CagdasOzgenc to your own post.
You may find a number of relevant posts arriving to similar conclusions in this list. Perhaps some of them will be formulated more clearly than I have managed.
answered Jan 15, 2016 at 18:41
$\begingroup$
$\endgroup$
2
No. The absolute value of AIC indicate nothing. The residuals are included in AIC calculation, so when you transform your data your residuals will be "transformed" too (actually, you will calculate new residuals for transformed data, and they are incomparable with your initial residuals).
It can be seen, i.e., from this guide, chapter "MINIMIZING THE LOSS OF INFORMATION".
-
$\begingroup$ The point about the absolute value of AIC is off-topic because the question is about comparing several AIC values. I am not sure how relevant the point regarding residuals is in its present formulation. Instead, the accent should be on the dependent variable and its likelihood. $\endgroup$ Commented Jan 15, 2016 at 18:42
-
$\begingroup$ @RichardHardy agree with you. The words about absolute values were not clear, I should elaborate more on this, I wanted to say smth similar to the post, link on which you provided below. $\endgroup$ Commented Jan 16, 2016 at 13:52