8
$\begingroup$

I would like to evaluate the predictive performance of a statistical model using Normalized Root Mean Square Error (NRMSE = RMSE/mean (observed)). However, the mean value of the observation data is all '0' (all observed data are '0'). How can I calculate the NRMSE, otherwise is there any other technique which can represent the relative/percentage value of error with such data aspect?

Improve this question
$\endgroup$

5 Answers 5

Reset to default
9
$\begingroup$

I think you can, but instead of dividing the RMSE by the mean, you may divide it by (max-min) value

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Can you extend your answer? We can also transform it to a comment. $\endgroup$
    – Ferdi
    Mar 29, 2017 at 8:43
5
$\begingroup$

I think Euan has a right answer. There are ways to calculate the NRMSE, RMSE/(max()-min()) and RMSE/mean(). You should know which is better to be used in your case. For example, when you are calculating the NRMSE of a house appliance, it is better to use the RMSE/(max()-min()). Because in this way it can show the NRMSE when the appliance is running. The reason why your mean value is 0 could be the data has both positive part and negative part, therefore, I think RMSE/(max()-min()) can show how your data spreads.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ In the question, he says that all observed data points is 0, so that doesn't actually work. $\endgroup$
    – Fabian Ying
    Mar 22, 2018 at 15:25
1
$\begingroup$

Euan Russano suggests dividing by the range of observations which is common (e.g. https://en.wikipedia.org/wiki/Root-mean-square_deviation NRMSD). But this would still be dividing by zero in your case because the range of observations is zero.

other measures of association (like correlation) will also be undefined because the variance is zero.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Was the OP actually interested in some form of correlation? it seems that if all observations are 0 there is no way to estimate any form of variation. This is the same for any constant c. $\endgroup$
    – Michael R. Chernick
    Feb 3, 2018 at 3:44
  • $\begingroup$ The way I get around the zeros issue is by forcing a 1 if the range is 0. $\endgroup$
    – marcotama
    Sep 17, 2018 at 6:34
0
$\begingroup$

You would normally divide by a measure of "spread". Either max(obs)-min(obs), as already mentioned, or directly the standard deviation of your observations, which is preferred for normally (or quasi-) distributed data. This is objective and gives your NRMSE nice units of "standard deviations of the observed data". You could also divide by the variance. If your observations are not constant, these two quantities should not be zero. Hope it helps.

Improve this answer
$\endgroup$
0
$\begingroup$

Normalizing should be performed depending on the reference value. So, for the forecasting studies, since we are trying to approach the real value, you may consider dividing by the real value.

$$\textrm{RMSE} = \sqrt{1/n\sum(y - y_i)^2/n},~ i = 1,\ldots,n $$

$$\textrm{NRMSE} = \textrm{RMSE}/y $$

Keep in mind that if you have only one sample then RMSE would be a wrong choice. Let's say the real value is 80, and the approximation is 60. If you apply RMSE, it will give you the difference between those values, not the percentage error. That is: $$\textrm{RMSE} = \sqrt{(80-60)^2/1}= 20.$$

However, $\textrm{NRMSE }$ will give you the error as a percentage: $\textrm{NRMSE }= 20/80 = 1/4 = 25\%. $

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.