2
$\begingroup$

I have an extremely noisy dataset, and I try to minimize the MAE. Apparently, when my linear regression makes its prediction - it has something like -1000 MAE (when compared to the MAE of an all-zero vector), and as I divide the estimation of every observation by some factor, it improves dramatically:

 - factor 1.5 | mae: -458.303432094
 - factor 2   | mae: -211.376918945
 - factor 3   | mae:  -50.0865186752
 - factor 4   | mae:   -2.34636554994
 - factor 5   | mae:   15.2679777338
 - factor 7   | mae:   25.0762924364
 - factor 10  | mae:   25.0904607219
 - factor 13  | mae:   22.3588382822
 - factor 15  | mae:   20.5381352161
 - factor 20  | mae:   16.7937150084

Note: the MAE is sometime negative because it is compared to the MAE of an all-zero vector. So -1000 means that the MAE of an all-zero vector is better than the algorithm's prediction by a 1000. My model does only slightly better than an all-zero vector when I divide my model's estimation by 10.

Why is it that my linear regression guesses numbers that are so big, such that they harm the MAE score so significantly? Is there any good method to deal with that?

edit: important to note that this model is not overfitted or so. The results reflect reality - dividing by 10 does better in both validation and test set.

$\endgroup$
  • 1
    $\begingroup$ How exactly did you get those values? How MAE, that is mean of absolute values, could be negative..?! $\endgroup$ – Tim Oct 30 '17 at 12:14
  • $\begingroup$ Note that it is compared to the MAE of an all-zero vector (I pointed it out but I'll edit to make it more clear) $\endgroup$ – dan Oct 30 '17 at 13:27
  • $\begingroup$ I wouldn't call it Jungian, but in our culture 'bigger is better', so I think it would be less confusing if you subtracted the MAE of the zero guess from the MAE of your model, so the number you presented showed the improvement of the model over the guess of 'all zero'. $\endgroup$ – aginensky Oct 30 '17 at 13:35
  • $\begingroup$ It does! That's exactly the problem. My model is significantly worse than an all-zero vector when I do not divide my prediction $\endgroup$ – dan Oct 30 '17 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.