4
$\begingroup$

Bias is defined as an average of all errors (without abs) and this is, IMO, what I want.

However, I have been asked to give MEAN ERROR. Is this the same than bias and is it wrong to call bias as mean error?

Just in case I’m messing these definitions totally, description what I try to do: Positions are forecasted and compared to absolute values. To give the error, RMSE and BIAS are calculated.

$\endgroup$
  • 2
    $\begingroup$ Roughly speaking, bias is "mean error", but as @abaumann mentioned, "mean error" is not the same as "mean squared error". $\endgroup$ – Michael M Apr 9 '14 at 13:21
  • $\begingroup$ Sorry, I'm engineer so I dont't really get that. Where did I hint mean error = mean squared error? Edit: So when roughly speaking I could use mean error, but not when I pretend to know what I'm saying? $\endgroup$ – braveslisce Apr 9 '14 at 18:24
  • $\begingroup$ Bias can be computed with respect to any measure of central tendency (usually mean/expectation, rarely median), that's why I started with 'roughly'. Bias wrt mean is the same as mean error. $\endgroup$ – Michael M Apr 9 '14 at 19:47
  • $\begingroup$ I’m really glad that you went through the trouble of answering, thank you! $\endgroup$ – braveslisce Apr 10 '14 at 6:57
  • 1
    $\begingroup$ Is it possible that for "mean error" they mean "mean absolute error"? In other words, if my errors on 3 observations are -10, 8, and 2 the mean error is 0 (as others note, 0 mean error is a characteristic of lots of models), but the mean absolute error is 20/3 = 6.7, so on average you are 6.7 units off. $\endgroup$ – zbicyclist Dec 25 '15 at 5:10
3
$\begingroup$

Very briefly, the MSE is the second moment of the bias. Let $\hat{\theta}$ be an estimator for the true quantity $\theta$. Then we have that

$Bias_\theta(\hat{\theta})=E(\hat{\theta})-\theta=E(\hat{\theta}-\theta)$

while

$MSE_\theta(\hat{\theta)} = E((\hat{\theta}-\theta)^2)$ and $RMSE \equiv \sqrt(MSE)$.

$\endgroup$
  • $\begingroup$ Hi, unfortunately I don’t really get that, that is my fault and I will try to learn more. What I’ get from your answer is that Bias is a function of predicted value and measured value – but honestly I think that is given. Second line tells that RMSE is a square root is a same function where error is powered. This will lead to the fact that RMSE and Bias depends each other which can’t obviously is not true. And finally, how does this answer to my question? Sorry to be thick $\endgroup$ – braveslisce Apr 9 '14 at 19:09
  • 1
    $\begingroup$ Mean squared error is variance plus squared bias. So there is clearly a relation between the two guys. $\endgroup$ – Michael M Apr 9 '14 at 19:48
  • $\begingroup$ The formula of RMSE is same than for standard deviation -> that would kinda of say that they are the same. Which would mean RMSE is a measurement of precision, and bias should be a measurement of accuracy. There can't be relation. $\endgroup$ – braveslisce Apr 9 '14 at 20:00
  • 1
    $\begingroup$ Why not? Think about it. When we call a estimator "unbiased", we mean that on average, its estimate is equal to the true value of theta. However, there can still be a lot of variation around the true value, so that the error of the estimate is high. The James-Stein estimator is a biased estimator with lower MSE than OLS, for example. $\endgroup$ – abaumann Apr 10 '14 at 9:02
  • $\begingroup$ Yes, you are right. Even though not possible for me to think it without too much work, it was easy to test with data. RMSE equals to standard deviation only when bias is removed. This leads, imo, against current way in our field to express errors with RMSE and BIAS. Much more telling would be bias or mean error and standard deviation of the error. Thank you for making me rechecks something I thought I knew. $\endgroup$ – braveslisce Apr 11 '14 at 12:31
3
$\begingroup$

In plain English, in a model a Mean Error is typically equal to zero. Otherwise, you have a really bad model. Let's face it the simplest model is a "naive" model where you simply take the average of all your values. Such a simple model would already have a Mean Error = 0. If you have constructed a more complex model, and its Mean Error is different than zero, than your model clearly has a "bias." If your model's Mean Error is positive it has an upward bias (it overestimates the actual values). If your model's Mean Error is negative it has a downward bias (it underestimates the actual values). This is not to be confused with the Mean Absolute Error (MAE) of your model and the Root Mean Squared Error (RMSE) also called Standard Error of your estimate. The vast majority of models do have an MAE and an RMSE, but most often do not have a Mean Error (or bias).

$\endgroup$
  • $\begingroup$ @Gaetan_Lion, that is true of the model fit data. The mean error outside the model (on a holdout sample, or of a forecast) is not zero. $\endgroup$ – zbicyclist Dec 25 '15 at 5:13
  • 1
    $\begingroup$ @zbicyclist I agree with you. And, that is true for any model in any circumstances. And, it may not have that much bearing on the quality of the model's original structure. It may not depict an inherent bias, but simply a temporal divergence between actuals and model estimates. $\endgroup$ – Sympa Dec 25 '15 at 20:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.