I've been doing a machine learning competition where they use RMSLE (Root Mean Squared Logarithmic Error) to evaluate the performance predicting the sale price of a category of equipment. The problem is I'm not sure how to interpret the success of my final result.

For example if I achieved a RMSLE of $1.052$ could I raise it the the exponential power $e$ and interpret it like rmse? (ie. $e^{1.052}=2.863=RMSE$)?

Could I then say that my predictions were $\pm \$2.863$ on average from the the actual prices? Or is there a better way to interpret the metric? Or can the metric even be interpreted at all with the exception of comparing to the other RMSLEs of other models?

  • With my limited knowledge, it's to: 1. to remove heteroscedasticity 2. to solve the problem of different dimensions – user35860 Dec 8 '13 at 16:51

I haven't seen RMSLE before, but I'm assuming it's $\sqrt{ \frac{1}{N} \sum_{i=1}^N (\log(x_i) - \log(y_i))^2 }$.

Thus exponentiating it won't give you RMSE, it'll give you

$e^\sqrt{ \frac{1}{N} \sum_{i=1}^N (\log(x_i) - \log(y_i))^2 } \ne \sqrt{\frac{1}{N} \sum_{i=1}^N (x_i - y_i)^2}$.

If we take the log of both sides, we get the RMSLE versus $\frac{1}{2} \log \left( \frac{1}{N} \sum_{i=1}^N (x_i - y_i)^2 \right)$, which is clearly not the same thing.

Unfortunately, there isn't a good easy relationship in general (though someone smarter than me / thinking about it harder than me could probably use Jensen's inequality to figure out some relationship between the two).

It is, of course, the RMSE of the log-transformed variable, for what that's worth. If you want a rough sense of the spread of the distribution, you can instead get a rough sense of the spread of their logarithm, so that a RMSLE of 1.052 means that the "average" is $2.86$ times as big as the true value, or 1/2.86. Of course that's not quite what RMSE means....

  • Hi @Dougal thanks! this definitely helps clear things up. – Opus Apr 20 '13 at 14:13

I don't know if there is a straightforward generic interpretation, even analysing a particular case.

For example, you may be interested in evaluating what would be the error if you predict all the cases with the mean value and compare it to your approach.

Anyway, I believe RMSLE is usually used when you don't want to penalize huge differences in the predicted and true values when both predicted and true values are huge numbers. In these cases only the percentual differences matter since you can rewrite

$\log{P_i + 1} - \log{A_i +1} = \log{\frac{P_i + 1}{A_i +1}}$.

For example for P = 1000 and A = 500 would give you the roughly same error as when P = 100000 and A = 50000.

My understanding is, when we do logarithm both on prediction and actual numbers, we'll get much smoother results than original one. And reduce the impact of larger x, while emphasize of smaller x for $\log{x+1}$.

Also you'll get a intuitive impression by drawing a simple graph of $y=\log{x+1}$.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.