goodness of fit log transformed vs not log transformed

Question

I have a relationship of two variables which is somehow log shaped. Now, I establish two models for this dataset, for one I log transform the dependent variable:

    y_ln=ln(y)
    fit=lm(y~x) #lm fits the linear model
    fit_ln=lm(y_ln~x)

If fit_ln has a higher R², is it valid to say that the fit_ln model is the better one?

Answer 1

Nope, that will not work. Because the outcome is not the same, and the scaling is not linear.

There are many ways to get a measure, but the one suggested by Wooldridge is to calculate the square correlation af $y$ (in levels) and the fitted value $\hat{y}$ obtained from the regression - after a back transformation.

Now the question is how to obtain a prediction when $\log(y)$ is the dependent variable. Wooldridge suggests (but there are many ways to do this):

1. Obtain the fitted values from $\log(y)$ on $x_1, x_2, ... , x_n$, as well as the residuals $u_i$.

2. Calculate: $a_0 = n^{-1} \sum_{i=1}^n \exp(u_i)$

3. Then for each $y_i$ calculate: $\hat{y}_i = a_0 \cdot \exp(\widehat{\log(y)})$

Now you can calculate an $R^2$ which is comparable.

Answer 2

Consistent with @repmat, it is also a good idea to compute accuracy scores that were not the scores used to optimize either of the two models. See for example median absolute error on the original scale.

But related to the bigger picture, analysts often seem too quick to assume that either log or original scale "works", and that if using long the number to subtract before taking logs is 0.0. These are tenuous assumptions. Think about using either a transformation-free method (semiparametric ordinal regression models such as the proportional odds model, with no binning of Y) or nonparametric transform-both-sides additive models. These are both covered in RMS. Semiparametric models also have the advantage of being robust to outliers.