Should I convert the number after log transformation back to the original for calculating RMSE?

Question

When I built the model, I applied the log transformation to all variables including the dependent variables. Now, I'm calculating the RMSE for the evaluation, and the result is in the log format, which is pretty small, around 0.48.

However, I don't think the number is correct to evaluate the model under this situation. For example, if one actual value is 10.1 after log transformation, and its predicted value is 10; another actual value is 1.1, and its predicted value is 1, the residuals under the log transformation of these two numbers are both 0.1. But if I convert them back to the original format and calculate the residuals: the residuals of exp(10.1)-exp(10) is way too different from the exp(1.1)-exp(1).

My question is: How to I fix this problem? (ex.manually calculate the RMSE after converting the predicted value in log format to the original format?)

Answer 1

Whether it makes more sense to evaluate your predictions on the original or on the log scale depends on your application, or on your loss function. Essentially: use a loss function that correctly measures how painful each loss is. (Which is two ways of saying the same thing.) We statisticians here can't tell you this. It depends on your domain.

---

Note that you have at least one error in your analysis.

• If you want an unbiased prediction, then the RMSE is the correct evaluation tool, since it will be minimized (in expectation) by an unbiased prediction. However, an unbiased prediction $\hat{y}$ on the log scale turns into a biased prediction on the original scale if you simply exponentiate, $e^{\hat{y}}$. You need a bias correction term, $e^{\hat{y}+\frac{\hat{\sigma}^2}{2}}$. (Yes, this also holds for non-time series predictions.)
• If you want a point prediction of the conditional median, then you can simply exponentiate, $e^{\hat{y}}$, to turn a median prediction on the log scale into a median prediction on the original scale (because exponentiation is monotone). However, then the RMSE is not an appropriate measure any more, at least on the original scale. You should use the MAE, which is minimized in expectation by a median prediction.

More information here and here and here.

Answer 2

As @StephanKolassa says, this depends on your application.*

One important thing to keep in mind: log transforms make a lot of sense when errors are proportional to the values, as this page discusses. If you are primarily interested in relative (or percentage) errors, staying in the log scale might be best. In your example, the difference between 10.1 and 10.0 on (natural) log-transformed data is just over 10% in the original scale, the same percentage difference in the original scale as there is between 1.1 and 1.0 on the log-transformed scale.

Note that your RMSE in the log scale of 0.48 isn't actually small at all if the dependent variable was log-transformed; exp(0.48) = 1.62, or a 62% error!

If your interest is in errors on the original scale of measurement, however, then you will either (1) have to use the types of approaches that @StephanKolassa recommends or (2) redo the analysis in the original scale of measurement of the dependent variable. Given the 62% RMSE error from your model with all variables log-transformed, you should revisit your approach to the modeling.

---

*The need of your application should always inform your choice of a loss function. Look at the documentation for the packages you used to see what they choose as default loss functions if you don't specify one yourself. As it is increasingly easy to use powerful statistical tools, it's very important to know what hidden assumptions those tools are making about your data before you rely on them.