1
$\begingroup$

How to determine whether a mean squared error is low or high? For example: In my linear regression problem if I get mean squared error as 21.67, how do I decide whether the error is low or high? Is there a bench mark ??

$\endgroup$
14
  • $\begingroup$ Mean absolute deviation (MAD), or mean absolute error (MAE), would be easier to interpret as they use the same scale as the data itself. Also, you could look at $1-R^2$ or $1-R^2_{adj.}$ which also indicates how large your errors are as compared with the data itself. $\endgroup$ – Richard Hardy Jan 15 '15 at 10:38
  • $\begingroup$ @RichardHardy what about residual sum of squares?? $\endgroup$ – Elizabeth Susan Joseph Jan 15 '15 at 10:41
  • $\begingroup$ I think it shares the same problem as MSE has; it is a relative measure and is also measured on a scale different than that of the data. After all, $RSS=MSE*N$ where $N$ is the number of data points. $\endgroup$ – Richard Hardy Jan 15 '15 at 10:49
  • $\begingroup$ @RichardHardy - thats great, This is the first time I came across a team called mean absoulte deviation. So is there any resources to learn more about it? and I am implementing linear regression in python $\endgroup$ – Elizabeth Susan Joseph Jan 15 '15 at 10:54
  • $\begingroup$ MAD is quite a simple thing: take the absolute values of all errors and calculate the mean. In MSE, you square the errors first and then calculate the mean, whereas in MAD you take absolute values instead of squaring. Consequently, the interpretation is as straightforward as it can be. There must hundreds of sources from which you can learn about MAD, cannot recommend any particular one. Regarding Python - sorry, I have no experience with it (I use R). But once you have your errors, calculating MAD manually is very simple. $\endgroup$ – Richard Hardy Jan 15 '15 at 11:02
1
$\begingroup$

MSE is a relative measure. If $y_i$ is your data point and $\hat{y}_i$ is an estimate for this data point, then MSE is:

$$MSE = \frac{1}{N} \sum^N_{i=1} \left( \hat{y}_i - y_i \right)^2$$

If $y$ is measured in meters it will give different results than if it is measured in kilometers etc. You can read more about similar measures of fit in here.

So MSE is low or high comparing to some other model.

$\endgroup$
7
  • $\begingroup$ I was doing an housing prices problem in which I got an mse of 24. So how do I compare it to some other model? can you be more specific. I am still confused. $\endgroup$ – Elizabeth Susan Joseph Jan 15 '15 at 10:39
  • 1
    $\begingroup$ Why do you insist on comparing models using MSE? You could use something else instead (as I suggested in a comment to the original post). $\endgroup$ – Richard Hardy Jan 15 '15 at 10:51
  • $\begingroup$ @ElizabethSusanJoseph as Richard said - why do you insist on MSE? MSE won't tell you that your model is "correct", you could use it just to say if it is worse (or better) than other model. $\endgroup$ – Tim Jan 15 '15 at 11:02
  • $\begingroup$ But yes, you can compare two models using MSE; the model with the smaller MSE produces fitted values that are closer to the true values (which is what you want). However, be careful; you can always build a very rich model that has a very low MSE but that won't mean your rich model is necessarily better than a parsimonious model with larger MSE. How to go about that? It's a broad topic... :) $\endgroup$ – Richard Hardy Jan 15 '15 at 11:06
  • $\begingroup$ @Tim - I was working through a problem, in that problem they were calculating mean squared error. this is the first time I am implementing linear regression. $\endgroup$ – Elizabeth Susan Joseph Jan 15 '15 at 11:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.