1
$\begingroup$

I'm working on a Machine Learning regression task, where I have to predict a continuous value, and I have to minimize the Mean Absolute Percentage error, the problem that the target values ranges from zero to a bigger values, but most of the values are closer to zero, and when calculating the MAPE, it explodes due to small numbers in the denominator, so I came up with idea of shifting the values by some big amount, like adding the value 100 to each of the target values, and after training I can subtract 100 from the predictions to get the real value, I tried it and it works well, but is that right?

$$ MAPE = \frac{1}{N}\sum_{t=1}^{n}\left| \frac{At - Ft}{At} \right| $$

Improve this question
$\endgroup$
3
  • $\begingroup$ Welcome to Cross Validated! $//$ $1)$ Note the issues with MAPE. $//$ $2)$ If you are able to achieve (reliable) strong predictive performance, quantified by your measure of performance of interest, what else is there? $//$ $3)$ I wonder if it would be possible to treat that $100$ as a hyperparameter to tune. Sure, you get good performance by adding/subtracting $100$, but what about if you use $10$ or $10000?$ (Perhaps you have already tried various values.) $\endgroup$
    – Dave
    Commented Oct 5, 2023 at 14:16
  • $\begingroup$ hi @Dave , thanks for the response, I didn't try other values than 100, but I thing if add more bigger values like 10000, the mape will decrease, the thing is, is adding or shifting the target values for training, and then subtracting the shifting value from the predictions, is this statistically or mathematically right to do, even if it's improve the model performance ? $\endgroup$
    – Mohamed Essam
    Commented Oct 5, 2023 at 14:27
  • $\begingroup$ Do you have any values of zero in the original data? $\endgroup$
    – Dave
    Commented Oct 5, 2023 at 17:24

1 Answer 1

Reset to default
1
$\begingroup$

There are reasons not to use MAPE. However, if that is what you have to optimize (for whatever reason), then your goal is to do as well as possible in terms of MAPE.

The statistical/mathematical problem can be thought of as some kind of minimization of MAPE over many possible ways of making predictions. You've found a way to make predictions that gives a good MAPE score. While this might not be the optimal way to make predictions, you seem to have an effective model.

It might be that $100$ is fairly specific to your particular data, so standard validation procedures would be warranted. For instance, if you bootstrap your data and find the optimal value to be $100$ sometimes but that, other times, $100$ is a terrible value and the good values to pick are in the thousands, then you might not believe this modeling approach to be reliable.

A drawback to kind of "hacking" around like this is that it is not driven by a classical theory, so standard theorems do not apply. However, computational methods (like the bootstrap mentioned above) can lead to convincing evidence.

Basically anything can be an estimator, and while there are estimation techniques that have desirable properties under particular assumptions (e.g., maximum likelihood) that might be our go-to methods because they tend to work well, we don't shy away from effective alternatives (if we put together evidence that they are, indeed, effective).

Improve this answer
$\endgroup$
3
  • $\begingroup$ thanks for you answer, I tried other values than 100, when I tried bigger values like 1000 and 10000, the model was worsening, and I tried smaller values like 10 and 1, and it was better in terms of speed of convergence and better predictions, so I don't know what can I conclude from these results, but I think that shifting the target values by any value greater than zero, but not a very big value, led to better results. So what are your thoughts regarding these results ? $\endgroup$
    – Mohamed Essam
    Commented Oct 5, 2023 at 15:30
  • $\begingroup$ @MohamedEssam This sounds like a hyperparameter to tune, not really different from tuning a regularization hyperparameter to get the best out-of-sample performance. // I don't know the circumstances that would lead to this, but I could imagine that you will keep getting better performance as you keep decreasing the value toward zero, only for the performance to drop off at exactly zero. $\endgroup$
    – Dave
    Commented Oct 5, 2023 at 15:35
  • $\begingroup$ I very much second Dave's recommendation to read this thread, as well as his answer to basically treat this as a ML task with the MAPE as the loss function. Much better to use the loss function you actually want to optimize (the MAPE), rather than to tweak the data and use a different loss function (presumably something like the square loss that is usually built in). I also recommend Kolassa (2020), because I do believe it directly addresses your issues. $\endgroup$
    – Stephan Kolassa
    Commented Oct 6, 2023 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.