0
$\begingroup$

I have a multioutput regression model that predicts float values. When using MAPE to evaluate regression model performance (using either built in libraries or implementing a function for it) I am getting very large numbers

from sklearn.metrics import mean_absolute_percentage_error
...
xtrain, xtest, ytrain, ytest = train_test_split(X, Y, test_size=0.1)
...
print("Mean Absolute Percentage Error: ", mean_absolute_percentage_error(ytest, model.predict(xtest)))

Would give me a very large number:

Mean Absolute Percentage Error: 43288238481823123

Another implementation would provide me infinity

def mean_absolute_percentage_error(y_true, y_pred): 
    y_true, y_pred = np.array(y_true), np.array(y_pred)
    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100

print("Mean Absolute Percentage Error: ", mean_absolute_percentage_error(ytest, model.predict(xtest)))

Mean Absolute Percentage Error: inf

In the same model the R^2 value of the model would be close to 1.

I am posting this question to ask if MAPE has strong limitations or scenarios that could lead to these results.

I can provide the dataset and multioutput algorithm if needed.

$\endgroup$

1 Answer 1

4
$\begingroup$

MAPE can be problematic (see this thread for MAPE's shortcomings).

In scikit-learn, if one of the values of y_true is zero, that will result in an arbitrarily high number.

Another example of that is the following (Source)

>>> from sklearn.metrics import mean_absolute_percentage_error
>>> y_true = [1., 0., 2.4, 7.]
>>> y_pred = [1.2, 0.1, 2.4, 8.]
>>> mean_absolute_percentage_error(y_true, y_pred)
112589990684262.48

For additional documentation on scikit-learn's metric MAPE, go here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.