1
$\begingroup$

I have seen multiple questions and answers about this, but I haven't been able to understand, so, I'm gonna try to ask as simple as possible. I have built several models to forecast future value of a variable. When I test the model (on out of training data) I have received the RMSE obtained by each one.

Example:

RMSE: 0.05, Min value of dataset: 1, Max value of dataset: 2

How can I calculate the accuracy of the model? Sorry if the question is duplicated, thanks in advance.

$\endgroup$
4
  • $\begingroup$ Is this a classification or regression problem? $\endgroup$ – Dave Sep 8 '20 at 23:22
  • $\begingroup$ Regression problem $\endgroup$ – David Díaz Sep 8 '20 at 23:23
  • $\begingroup$ Then what do you mean by “accuracy”? $\endgroup$ – Dave Sep 8 '20 at 23:23
  • $\begingroup$ The effectiveness (in percentage), is it wrong? $\endgroup$ – David Díaz Sep 8 '20 at 23:26
1
$\begingroup$

Answering with a question: you would like to calculate "percentage" of what exactly? If you want to have a unitless measure, you can take a look at MAPE (see ), or $R^2$ (see ), but as you can learn from What are the shortcomings of the Mean Absolute Percentage Error (MAPE)? and Is $R^2$ useful or dangerous?, both have their shortcomings. Moreover, accuracy is a far from perfect measure of model performance, and there are many issues with it, so for classification problems this is often not the best choice for a metric as well.

TL;DR from the threads mentioned above is that for each "unitless" metric, you need to normalize it somehow and often how you normalize will influence the result and may lead to misguided interpretations.

For regression problems it is usually much more useful to have metrics using units of the thing that you are predicting, like mean absolute error, or root mean squared error, e.g. if you are predicting prices, they tell you by how many euros you are off, on average, etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.