4
$\begingroup$

I would like to evaluate the predictive performance of a statistical model using Normalized Root Mean Square Error (NRMSE = RMSE/mean (observed)). However, the mean value of the observation data is all '0' (all observed data are '0'). How can I calculate the NRMSE, otherwise is there any other technique which can represent the relative/percentage value of error with such data aspect?

$\endgroup$
4
$\begingroup$

I think you can, but instead of dividing the RMSE by the mean, you may divide it by (max-min) value

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Can you extend your answer? We can also transform it to a comment. $\endgroup$ – Ferdi Mar 29 '17 at 8:43
2
$\begingroup$

I think Euan has a right answer. There are ways to calculate the NRMSE, RMSE/(max()-min()) and RMSE/mean(). You should know which is better to be used in your case. For example, when you are calculating the NRMSE of a house appliance, it is better to use the RMSE/(max()-min()). Because in this way it can show the NRMSE when the appliance is running. The reason why your mean value is 0 could be the data has both positive part and negative part, therefore, I think RMSE/(max()-min()) can show how your data spreads.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ In the question, he says that all observed data points is 0, so that doesn't actually work. $\endgroup$ – Fabian Ying Mar 22 '18 at 15:25
1
$\begingroup$

Euan Russano suggests dividing by the range of observations which is common (e.g. https://en.wikipedia.org/wiki/Root-mean-square_deviation NRMSD). But this would still be dividing by zero in your case because the range of observations is zero.

other measures of association (like correlation) will also be undefined because the variance is zero.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Was the OP actually interested in some form of correlation? it seems that if all observations are 0 there is no way to estimate any form of variation. This is the same for any constant c. $\endgroup$ – Michael R. Chernick Feb 3 '18 at 3:44
  • $\begingroup$ The way I get around the zeros issue is by forcing a 1 if the range is 0. $\endgroup$ – marcotama Sep 17 '18 at 6:34
0
$\begingroup$

You would normally divide by a measure of "spread". Either max(obs)-min(obs), as already mentioned, or directly the standard deviation of your observations, which is preferred for normally (or quasi-) distributed data. This is objective and gives your NRMSE nice units of "standard deviations of the observed data". You could also divide by the variance. If your observations are not constant, these two quantities should not be zero. Hope it helps.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.