Comparing AIC of a model and its log-transformed version

Question

The essence of my question is this:

Let $Y \in \mathbb{R}^n$ be a multivariate normal random variable with mean $\mu$ and covariance matrix $\Sigma$. Let $Z := \log(Y)$, i.e. $Z_i = \log(Y_i), i \in \{1,\ldots,n\}$. How do I compare the AIC of a model fit to observed realizations of $Y$ versus a model fit to observed realizations of $Z$?

---

---

My initial and slightly longer question:

Let $Y \sim \mathcal{N}(\mu,\Sigma)$ be a multivariate normal random variable. If I want to compare a model fit to $Y$ versus a model fit to $\log(Y)$, I could look at their log-likelihoods. However, since these models aren't nested, I can't compare the log-likelihoods (and stuff like AIC, etc.) directly, but I have to transform them.

I know that if $X_1,\ldots,X_n$ are random variables with joint pdf $g(x_1,\ldots,x_n)$ and if $Y_i = t_i(X_1,\ldots,X_n)$ for one-to-one transformations $t_i$ and $i \in \{1,\ldots,n\}$, then the pdf of $Y_1,\ldots,Y_n$ is given by $$f(y_1,\ldots,y_n)=g(t_1^{-1}(y),\ldots,t_n^{-1}(y))\det(J)$$ where $J$ is the Jacobian associated with the transformation.

Do I simply have to use the transformation rule to compare

$$l(Y) = \log(\prod_{i=1}^{n}\phi(y_i;\mu,\Sigma))$$ to $$l(\log(Y))=\log(\prod_{i=1}^{n}\phi(\log(y_i);\mu,\Sigma))$$

or is there something else I can do?

---

[edit] Forgot to place logarithms in the last two expressions.

Answer 1

You cannot compare the AIC or BIC when fitting to two different data sets i.e. $Y$ and $Z$. You only can compare two models based on AIC or BIC just when fitting to the same data set. Have a look at Model Selection and Multi-model Inference: A Practical Information-theoretic Approach (Burnham and Anderson, 2004). They mentioned my answer on page 81 (section 2.11.3 Transformations of the Response Variable):

Investigators should be sure that all hypotheses are modeled using the same response variable (e.g., if the whole set of models were based on log(y), no problem would be created; it is the mixing of response variables that is incorrect).

And by the way, for using the AIC or BIC criteria, your models do not need to be necessarily nested (same ref, page 88, section 2.12.4 Nonnested Models), and actually that's one of the advantages of using BIC.

Answer 2

Akaike (1978, pg. 224) describes how the AIC can be adjusted in the presence of a transformed outcome variable to enable model comparison. He states: “the effect of transforming the variable is represented simply by the multiplication of the likelihood by the corresponding Jacobian to the AIC ... for the case of $\log\{y(n)+1\}$, it is −2 $\cdot \sum \log\{y(n)+1\}$, where the summation extends over $n = 1,2, ..., N$.”

Akaike, H. 1978. "On the likelihood of a time series model," Journal of the Royal Statistical Society, Series D (The Statistician), 27(3/4), pp. 217–235.

Answer 3

Here is an example that elaborates on Ben Bolker's response:

seedrates <- data.frame(rate = c(50, 75, 100, 125, 150), 
                        grain = c(21.2, 19.9, 19.2, 18.4, 17.9))
quad.lm <- lm(grain~poly(rate,2), data=seedrates)
loglin.lm <- lm(log(grain)~log(rate), data=seedrates)
oldopt <- options(digits=2)
AIC(quad.lm, loglin.lm)

          df   AIC
quad.lm    4  -4.1
loglin.lm  3 -37.2

We need to add sum(2*log(seedrates$grain)) = 29.6 to the AIC for the loglinear model (or, subtract it from the AIC for the quadratic model).

> AIC(quad.lm, loglin.lm) + matrix(ncol=2, c(0,0,0, sum(2*log(seedrates$grain))))
          df  AIC
quad.lm    4 -4.1
loglin.lm  3 -7.6

> options(oldopt)

This should probably be regarded as a toy example. With so few degrees of freedom to play with, it might just be argued that the loglinear model makes more sense. The quadratic model would imply that, a short distance beyond the upper limit of the rate, grain will start to increase.