0
$\begingroup$

Can I compare say two models where one have predictor P and the other have log(P) all else being equal?

I have found posts on the topic but I can only decipher from those that you cannot transform the response variable without adjustment, see here. But what about the predictors?

$\endgroup$

1 Answer 1

2
$\begingroup$

Yes, you can compare these two models using AIC. I presume you're asking because the models are not nested. AIC is a technique to estimate the minimum KL divergence between each candidate model from a set of candidate models and the true data-generating distribution, and it is not required that the set of candidate models be 'nested' within each other. See e.g. page 266-267 of Burnham and Anderson.

That said, you could use a Taylor expansion on $\log(P)$ to argue that they are approximately nested. Suppose that you're fitting linear models. Then the right-hand sides of your two models look like

\begin{aligned} \alpha_0 + \alpha_1P \end{aligned} and\begin{aligned} \beta_0 + \beta_1\log(P) &\approx \beta_0 + \beta_1(P-1) - \beta_1(1/2)(P-1)^2 + \beta_1(1/3)(P-1)^3 - \beta_1(1/4)(P-1)^4 \\ &= \beta_0^* + \beta_1^*P + \beta_2^*P^2+ \beta_3^*P^3+ \beta_4^*P^4 \end{aligned}

Thus, you're basically choosing between a simple linear regression or a polynomial linear regression, but with the very restrictive assumption that all of the coefficients share a particular 1-1 correspondence.

$\endgroup$
6
  • $\begingroup$ Thanks for your reply! I started experimenting to see if I could illustrate this in R. This is what I came up with, but I was expecting the fitted lines to be more similar. What am I doing wrong? $\endgroup$
    – Andreas
    Commented Jul 5, 2020 at 21:47
  • $\begingroup$ x <- rlnorm(100, 1) y <- rnorm(100, x, 5) plot(y~x) log.x <- log(x) f1 <- lm(y ~ log.x) f2 <- lm(y ~ poly(x, 4)) r <- seq(min(log.x), max(log.x), length.out=1000) s <- seq(min(x), max(x), length.out=1000) t <- predict(f1, newdata = data.frame(log.x = r)) u <- predict(f2, newdata = data.frame(x = s)) par(mfrow=c(1,2)) plot(y ~ log.x) lines(r, t, col = "red") plot(y ~ log.x) lines(log(s), u, col = "red") $\endgroup$
    – Andreas
    Commented Jul 5, 2020 at 21:48
  • $\begingroup$ That's pretty awful. How do I do line-breaks? $\endgroup$
    – Andreas
    Commented Jul 5, 2020 at 21:54
  • $\begingroup$ Re: the dissimilarity between the two plots you created. Your code fits an unstructured polynomial model, where each coefficient is allowed to be estimated independently, but the relationship will only hold when one constrains all of the coefficients to have the fixed relationship specified by the Taylor expansion. I think one would need to write a custom likelihood function to be optimized. $\endgroup$
    – psboonstra
    Commented Jul 8, 2020 at 12:52
  • $\begingroup$ Re: linebreaks. See here. It looks like you can use <br/> $\endgroup$
    – psboonstra
    Commented Jul 8, 2020 at 13:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.