# Conceptual understanding of root mean squared error and mean bias deviation

I would like to gain a conceptual understanding of Root Mean Squared Error (RMSE) and Mean Bias Deviation (MBD). Having calculated these measures for my own comparisons of data, I've often been perplexed to find that the RMSE is high (for example, 100 kg), whereas the MBD is low (for example, less than 1%).

More specifically, I am looking for a reference (not online) that lists and discusses the mathematics of these measures. What is the normally accepted way to calculate these two measures, and how should I report them in a journal article paper?

It would be really helpful in the context of this post to have a "toy" dataset that can be used to describe the calculation of these two measures.

For example, suppose that I am to find the mass (in kg) of 200 widgets produced by an assembly line. I also have a mathematical model that will attempt to predict the mass of these widgets. The model doesn't have to be empirical, and it can be physically-based. I compute the RMSE and the MBD between the actual measurements and the model, finding that the RMSE is 100 kg and the MBD is 1%. What does this mean conceptually, and how would I interpret this result?

Now suppose that I find from the outcome of this experiment that the RMSE is 10 kg, and the MBD is 80%. What does this mean, and what can I say about this experiment?

What is the meaning of these measures, and what do the two of them (taken together) imply? What additional information does the MBD give when considered with the RMSE?

-
Have you looked around our site, Nicholas? Consider starting at stats.stackexchange.com/a/17545 and then explore some of the tags I have added to your question. –  whuber May 29 '12 at 13:48
@whuber: Thanks whuber!. I've looked around the site, but to me I am still finding it a bit challenging to understand what is really meant in the context of my own research. –  Nicholas Kinar May 29 '12 at 15:19
add comment

## 2 Answers

I think these concepts are easy to explain. So I would rather just describe it here. I am sure many elementary statistics books cover this including my book "The Essentials of Biostatistics for Physicians, Nurses and Clinicians."

Think of a target with a bulls-eye in the middle. The mean square error represent the average squared distance from an arrow shot on the target and the center. Now if your arrows scatter evenly arround the center then the shooter has no aiming bias and the mean square error is the same as the variance.

But in general the arrows can scatter around a point away from the target. The average squared distance of the arrows from the center of the arrows is the variance. This center could be looked at as the shooters aim point. The distance from this shooters center or aimpoint to the center of the target is the absolute value of the bias.

Thinking of a right triangle where the square of the hypotenuse is the sum of the sqaures of the two sides. So a squared distance from the arrow to the target is the square of the distance from the arrow to the aim point and the square of the distance between the center of the target and the aimpoint. Averaging all these square distances gives the mean square error as the sum of the bias squared and the variance.

-
Thank you; this is very much appreciated. I am still finding it a little bit challenging to understand what is the difference between RMSE and MBD. As I understand it, RMSE quantifies how close a model is to experimental data, but what is the role of MBD? Maybe my misunderstanding is just associated with terminology. –  Nicholas Kinar May 29 '12 at 15:16
The mean bias deviation as you call it is the bias term I described. It measures how far the aimpoint is away from the target. Bias contributes to making the shot inaccurate. –  Michael Chernick May 29 '12 at 15:21
Thanks again, Michael. So a high RMSE and a low MBD implies that it is a good model? –  Nicholas Kinar May 29 '12 at 15:32
No a high RMSE and a low MBD just says that the model is poor because of a large variance rather than a large bias. The RMSE is the number that decides how good the model is. –  Michael Chernick May 29 '12 at 15:45
Ah - okay, this is making sense to me now. So if the RMSE tells us how good the model is, then what would be the purpose of looking at both the RMSE and the MBD? –  Nicholas Kinar May 30 '12 at 1:59
show 2 more comments

As far I can understand, a RMSE give a more accurate value of the error between model and observed, however the BIAS, in addition to give a value of the error (less accurate than the RMSE), it can also determine if the model is positive bias or negative bias, if the model is underestimating or overestimating the observed values.

-
add comment