Accuracy percentage-wise of a regression model [duplicate]

Question

I would like to check in percentage the accuracy of my regression model. I know that normally accuracy is used as a metric for classification. I have evaluated my model based on r-squared and also plotted the y_pred versus y_test.

However, I find it easier to understand the prediction performance in terms of accuracy. Therefore, I would like to do something like this (based on the sklearn, metrics.mean_absolute_percentage_error) :

accuracy = 100 - np.mean(mean_absolute_percentage_error(y_test,y_pred))
print('Accuracy:', round(accuracy, 2), '%.')

Does it make sense, would the result reflect the performance of the regression model based on a percentage of accuracy?

Answer 1

$R^2$ and MAPE do not communicate the same result. $R^2$ elicits the conditional expectation, and MAPE elicits the (-1)-median. Those are two different functionals of the underlying distribution.

In my opinion (Kolassa, 2020), you should first think about which "one number summary" of your predictive distribution you would consider "good", and then use an error measure that is optimized in expectation by that functional. If you want to predict the conditional expectation, use the MSE. If you want the conditional median, use the MAE. If you want the conditional (-1)-median, use the MAPE.

As a corollary, your model very likely does not optimize on MAPE. I argue (again Kolassa, 2020) that it makes no sense to run, say, an OLS and calculate point predictions (which are expectation predictions, which minimize the MSE in expectation), and then evaluate these predictions using a different evaluation measure. Instead, if you want MAPE-optimal predictions, use the MAPE as an objective function in your fitting step, as well - similar to how quantile regression uses a quantile loss to predict conditional quantiles.