0
$\begingroup$

First of all sorry for the bad title, but unfortunately, I can't think of a better one at the moment. Hopefully, that will change when my question is answered.

Let's say I have two sets of values: The target values $T_i$ and the actual values $X_i$ and I want to quantify the errors $E_i=X_i-T_i$ using a single number. Due to the fact that I don't want negative and positive to compensate each other and the fact that I'd like to punish outliers, I decided to use the Root Mean Square Error $\text{RMSE}$.

Now the question: How can I quantify the fluctuation of the errors in a way that can be compared with the $\text{RMSE}$? My first idea was to use the standard deviation. After all, the $\text{RMSE}$ is a mean value and it is a plausible assumption, that the errors $E_i$ are normally distributed. However, I think the standard deviation is "incompatible" with the $\text{RMSE}$, because unlike the $\text{RMSE}$ it does not operate on the squared errors.

What are your recommendations?

EDIT: I'll try to make things clearer. What I want to do is to quantify the error of the $\text{RMSE}$. After all it is a mean, so there must be a way to tell how much the values, this mean is based on, fluctuate. Let's say I would use the $\text{MAE}$ instead of the $\text{RMSE}$. Then I could easily quantify the fluctuation of the absolute errors using their standard deviation. However, to apply the same approach to the $\text{RMSE}$, I would have to calculate the square root of the standard deviation of the squared errors, which seems a little odd to me.

$\endgroup$
0
$\begingroup$

From what I can make out, you have two requirements:

  1. Ensure that positive and negative errors don't cancel out. So, you square the errors.
  2. You want to punish the models which have large errors for some predictions (but possibly lower errors elsewhere) more than models which have no large outlier errors(but possibly higher errors elsewhere). This is done by, again, squaring the errors. The square really amplifies the outliers. (For example, a model with errors 10 and 0 has squared-sum-of-errors of 100 whereas a model with errors 5 and 5 has squared-sum-of-errors of 50).

If you notice, both your requirements imply the use of squared errors. Whether or not you take the root after summing the squared-errors has no impact on meeting the above requirements.

$\endgroup$
  • $\begingroup$ Thank you for your quick reply. Apparently, I did not describe my interest well enough. What I want to do is to quantify the error of the RMSE. After all it is a mean, so there must be a way to tell how much the values, this mean is based on, fluctuate. I'll edit my question a little to make things clearer. $\endgroup$ – Nos Jun 18 at 7:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.