2
$\begingroup$

I'm building a machine learning model that realizes sales predictions based on a set of features, but for this specific problem it would not be important to have a spot-on prediction.

The problem is that with the MSE loss function I'm getting some predictions that get spot-on predictions on part of the validation data, and gets a somewhat high error on other points.

So, I was thinking if there is a established function that would help the algorithm prioritize models without grotesque errors.

Thanks in advance.

Improve this question
$\endgroup$
4
  • 5
    $\begingroup$ Yes: it's called MSE. It already penalizes larger errors more than smaller errors. What you have to decide--and you haven't supplied any information about this--is how much you want to penalize "grotesque" errors and how large "grotesque" actually is. $\endgroup$ – whuber♦ Sep 28 '17 at 18:40
  • $\begingroup$ Since MSE is not penalizing enough, one solution could be using MSE^2 ? $\endgroup$ – Lucas Sep 28 '17 at 19:01
  • $\begingroup$ Maybe, maybe not. Could be your model is not flexible enough, or your fitting method not robust enough, to achieve your desired precision with any loss function. $\endgroup$ – Paul Sep 28 '17 at 19:06
  • 3
    $\begingroup$ For many purposes MSE^2 is no different than MSE. The trick is to weight the differences of which it is comprised rather than weighting the final result. For instance--purely as a mathematical example rather than one with statistical meaning--you could take the mean fourth power rather than the mean square. But you have many more options than resorting to mere moments of that sort. Your focus ought to be on quantifying what you mean by "grotesque" and then quantitatively assessing just how bad such errors are compared to smaller ones. $\endgroup$ – whuber♦ Sep 28 '17 at 19:29
2
$\begingroup$

Often MSE actually penalizes the largest errors too much - being very wrong on a few outliers may be OK if the model is usually more or less right.

Since this is sales prediction, don't you have a loss function? i.e. the cost to the business of an error? On the face of things this is more likely to be the absolute error rather than the squared error.

It might be time to think about your descriptors. No loss function can make a good model if the input data is not predictive of the output.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.