I have an extremely noisy dataset, and I try to minimize the MAE. Apparently, when my linear regression makes its prediction - it has something like -1000 MAE (when compared to the MAE of an all-zero vector), and as I divide the estimation of every observation by some factor, it improves dramatically:
- factor 1.5 | mae: -458.303432094 - factor 2 | mae: -211.376918945 - factor 3 | mae: -50.0865186752 - factor 4 | mae: -2.34636554994 - factor 5 | mae: 15.2679777338 - factor 7 | mae: 25.0762924364 - factor 10 | mae: 25.0904607219 - factor 13 | mae: 22.3588382822 - factor 15 | mae: 20.5381352161 - factor 20 | mae: 16.7937150084
Note: the MAE is sometime negative because it is compared to the MAE of an all-zero vector. So -1000 means that the MAE of an all-zero vector is better than the algorithm's prediction by a 1000. My model does only slightly better than an all-zero vector when I divide my model's estimation by 10.
Why is it that my linear regression guesses numbers that are so big, such that they harm the MAE score so significantly? Is there any good method to deal with that?
edit: important to note that this model is not overfitted or so. The results reflect reality - dividing by 10 does better in both validation and test set.