I am currently building a regression MLP model to predict a target variable between [0,3]. The distribution for target variable is normal for the most part with a slight left skew. My model is excellent at predicting values >.1 with nearly 100% of predictions being within a 5% tolerable range, but for values <.1 the model has a sudden dropoff to where ~60% are within 5%. I tried doing an additive shift to the data, adding 1 for example so my new range is [1,4] and model is able to fit extremely well but when I transform back to the original scale I get stuck with poor performance. I have tried using MAPE, SMAPE, MSE, MAE, Huber, MSLE, with no major improvements in the low value areas. Why is it so difficult for MLPs to predict very small values and what potential changes can I do to my architecture/loss function/data to fit properly on the original scale?
1
-
$\begingroup$ A 5% error on a very small actual is very hard to achieve. If your actuals can vary from very small values, percentage errors are quite probably not what you want to use. Please take a look at the proposed duplicate. I would argue that it tells you that aiming for small percentage errors is mistaken in your case. $\endgroup$– Stephan KolassaCommented Aug 6 at 19:06
Add a comment
|