2
$\begingroup$

I've got a very small question. Say I'm making a linear regression model. When I test the model with a testing set, I get an MSE of 4.31 (arbitrary). What do I interpret from this? As in, what does this '4.31' represent, or rather mean?

Until now, I was thinking of it as the average squared difference between target and predicted values. Thus, I thought it showed that the average prediction differed by sqrt(4.31) from the average actual value, but I'm not sure.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
6
$\begingroup$

what does this '4.31' represent, or rather mean?

Until now, I was thinking of it as the average squared difference between target and predicted values.

This is exactly correct.

Thus, I thought it showed that the average prediction differed by sqrt(4.31) from the average actual value, but I'm not sure.

And this is incorrect.

That the mean squared error (MSE) is $4.31$ does not imply that the mean absolute error (MAE) is $\sqrt{4.31}$. In fact, there is no simple relationship between the MSE and the MAE that always holds.

Here is an example. Suppose your actuals are iid standard normally distributed, $y\sim N(0,1)$. You have two different predictions.

  • The first prediction is a flat zero, $\hat{y}=0$.

  • The second prediction is perfect, $\hat{y}=y$, with probability $0.9$, and off by $\sqrt{10}\approx 3.16$, $\hat{y}=y+\sqrt{10}$ with probability $0.1$.

    • The expected MSE of this is $0.1\times 10=1$, as above.
    • The expected MAE of this is $0.1\times\sqrt{10}\approx0.32$.

We see that you can have the same MSE but different MAEs.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks! Can't upvote because of lack of reputation. $\endgroup$
    – Mehul Jangir
    Jul 16, 2019 at 12:36
  • $\begingroup$ @stephankolassa How would you explain this difference to a non-technical manager? The average squared difference being 4.31 certainly makes it sound like the average difference is $\pm \sqrt{4.31}$! $\endgroup$
    – Dave
    Jul 16, 2019 at 15:15
  • $\begingroup$ @Dave: that's a very good question. Unfortunately, there is no easy answer. I would explain to him that it doesn't work that way for technical reasons and be ready with an example like this if the manager wants to dig deeper. Ultimately, I'm afraid the only solution is to demonstrate your competence to your manager to a degree that he trusts you even if what you say sounds unintuitive. $\endgroup$
    – Stephan Kolassa
    Jul 16, 2019 at 15:32
1
$\begingroup$

I'll just formulate some ideas and give leads to free introductory material.

The raw MSE figure in itself doesn't tell you much. Depending on the quantities in your sample space, sqrt(4.31) could be very small, or very big, which may make you want to use the model or not.

What you can use it for is the comparison of different models, and favor the one with smallest MSE. Within the data you have, your model with MSE = 4.31 could be the worst as well as the best.

From the Wikipedia Mean squared error page:

The MSE is a measure of the quality of an estimator—it is always non-negative, and values closer to zero are better.

The article contains further links about specifics on regression and prediction if you look for more advanced content.

For an accessible video introduction: Khan Academy has a serie on Squared Error

To sum it up: MSE is a relative measure of how well your model fits your sample data.

There is more to it if you know more about the distribution of the different variables, dependent, independent and residuals, and you might be interested in ANOVA and goodness of fit to go further on the topic.

I hope this helps, All the best,

Mathieu

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.