2
$\begingroup$

I have an extremely noisy dataset, and I try to minimize the MAE. Apparently, when my linear regression makes its prediction - it has something like -1000 MAE (when compared to the MAE of an all-zero vector), and as I divide the estimation of every observation by some factor, it improves dramatically:

 - factor 1.5 | mae: -458.303432094
 - factor 2   | mae: -211.376918945
 - factor 3   | mae:  -50.0865186752
 - factor 4   | mae:   -2.34636554994
 - factor 5   | mae:   15.2679777338
 - factor 7   | mae:   25.0762924364
 - factor 10  | mae:   25.0904607219
 - factor 13  | mae:   22.3588382822
 - factor 15  | mae:   20.5381352161
 - factor 20  | mae:   16.7937150084

Note: the MAE is sometime negative because it is compared to the MAE of an all-zero vector. So -1000 means that the MAE of an all-zero vector is better than the algorithm's prediction by a 1000. My model does only slightly better than an all-zero vector when I divide my model's estimation by 10.

Why is it that my linear regression guesses numbers that are so big, such that they harm the MAE score so significantly? Is there any good method to deal with that?

edit: important to note that this model is not overfitted or so. The results reflect reality - dividing by 10 does better in both validation and test set.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ How exactly did you get those values? How MAE, that is mean of absolute values, could be negative..?! $\endgroup$
    – Tim
    Commented Oct 30, 2017 at 12:14
  • $\begingroup$ Note that it is compared to the MAE of an all-zero vector (I pointed it out but I'll edit to make it more clear) $\endgroup$
    – dan
    Commented Oct 30, 2017 at 13:27
  • $\begingroup$ I wouldn't call it Jungian, but in our culture 'bigger is better', so I think it would be less confusing if you subtracted the MAE of the zero guess from the MAE of your model, so the number you presented showed the improvement of the model over the guess of 'all zero'. $\endgroup$
    – meh
    Commented Oct 30, 2017 at 13:35
  • $\begingroup$ It does! That's exactly the problem. My model is significantly worse than an all-zero vector when I do not divide my prediction $\endgroup$
    – dan
    Commented Oct 30, 2017 at 14:29

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.