5
$\begingroup$

I heard that adjusted R-squared from two model are comparable only if two models use the same response variables. So if the response variable in one model is Y and the other one is log(Y), then I should not use adjusted R-squared; rather I should use AIC, BIC, etc instead. Is this true?

$\endgroup$
  • $\begingroup$ You can't compare AIC or BIC of observations under different transformations either. They depend on the likelihood, which may be quite different on different transformation scales. In particular, in the case of regression models, AIC and BIC will be functions of the estimate of $\sigma^2$, which depends on the transformation. See here and here for some related questions. $\endgroup$ – Glen_b -Reinstate Monica Oct 9 '13 at 0:49
  • $\begingroup$ I should note that neither of those questions have answers either. $\endgroup$ – Glen_b -Reinstate Monica Oct 9 '13 at 1:08
  • 1
    $\begingroup$ Models can be used for varying purposes. Suppose that the goal is to have the narrowest (in the original, not-log scale) 95% prediction intervals. One way to argue that comparison using R-sq is a bad idea is to find a simple example of data such that the log-transformed version has a higher R-sq and yet produces wider confidence intervals. No such example comes to mind yet. $\endgroup$ – zkurtz Oct 9 '13 at 3:50
  • $\begingroup$ No, even without knowing the math behind the scene, if your proposed phenomenon is correct, then two models with the same set of predictors each predicting the original $y$ and $ln(y)$ will have the same AIC and BIC, which is not the case. $\endgroup$ – Penguin_Knight Oct 9 '13 at 14:07
2
$\begingroup$

I believe using R2 or adjusted R2 is okay in your case. Fact is that we should not use RSE(Residue Standard Error) when the scale is different. This is because both R2 and adjusted R2 are normalized quantity having maximum value of 1, but the RSE is not normalized.

Ref : https://datastoriesweb.wordpress.com/2017/01/15/interpreting-statistical-values/

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.