# Evaluation of log Vs. non log models

There are several posts on here about this question. The gist of them, as far as I understand, is that you cannot compare RMSE or MAE of two models where one is log transformed on the dependent variable since they are on different scales. So you can compare thousands of dollars to thousands of dollars but not log(thousands of dollars) to just thousands of dollars.

e.g.

Huge difference in regression standard error after log transformation of dependent variable

Back transforming regression results when modeling log(y)

Linear regression with log transformed data - large error

Compare Linear and Log standard error after transformation in R

I then found this article by Duke What's the bottom line? How to compare models.

In particular, this paragraph:

The root mean squared error and mean absolute error can only be compared between models whose errors are measured in the same units

(e.g., dollars, or constant dollars, or cases of beer sold, or whatever). If one model's errors are adjusted for inflation while those of another or not, or if one model's errors are in absolute units while another's are in logged units, their error measures cannot be directly compared. In such cases, you have to convert the errors of both models into comparable units before computing the various measures. This means converting the forecasts of one model to the same units as those of the other by unlogging or undeflating (or whatever), then subtracting those forecasts from actual values to obtain errors in comparable units, then computing statistics of those errors. You cannot get the same effect by merely unlogging or undeflating the error statistics themselves!

Does this mean that a sound approach to choosing between a log model and it's equivalent non log version would be to do e.g. cross validation and pick the model with the lowest RMSE or MAE after predicting on e.g. 5 folds?

Put another way, the questions I found while researching gave descriptive reasons why you cannot compare RMSE directly with initial model output, but if I understand if I just use the model to predict on test data, I can have a somewhat definitive answer to which model to select?

Is this a logical approach? A standard or typical approach? A good approach?

• If down voting can you let me know how to improve the question. Thanks Commented Apr 8, 2017 at 10:21
• I think there is a random down-voter at large. Sadly I cannot answer this question though but it seems useful so I up-voted it. Hope an expert passes by. Commented Apr 8, 2017 at 12:56
• I noticed that too, @mdewey. I've upvoted this question out of counter-spite. Commented Apr 9, 2017 at 1:37
• I don't see the relevance of cross-validation here in particular. The idea is that you have to compare apples to apples, whatever your metric. This is conceptually distinct from the process of comparing the performance of two models on test data. Edit: I take that back. I can see that performance on test data is important here, since one model could be directly optimizing the performance measure in-sample, while the other wouldn't. But it does still have to be apples-to-apples. Commented Apr 9, 2017 at 1:41
• @TheLaconic thanks for contributing to the discussion. You don't see the relevance of cross-validation here. OK. But I was not sure. That's why I asked a question on this q&a site... out of lack of understanding! Commented Apr 9, 2017 at 1:45

• Cool! Yes, exponentiating the $\log(\hat{y})$ will give provide estimates on the original scale of the $y$ variable. Residuals will be derived from these values. The link (you might have already seen here) has a nice example. Commented Apr 9, 2017 at 10:37