2
$\begingroup$

What is the general acceptable value of MAPE in industry ?. I am getting MAPE of around 24% on live data that has 48 data points in which 42 as train data and 6 as test data. I am trying to do ARIMA on these sets. This is a sales data that has both trend and seasonality components.

$\endgroup$

1 Answer 1

2
$\begingroup$

The question is meaningless. Yes, some organizations publish "benchmarks" for forecast accuracy, but these are completely useless, because achievable forecast accuracy depends on your data generating process, on the quality of your data, on your aggregation granularity (along multiple dimensions, including product, location, time and others) and on other factors.

For instance, slow movers are harder to forecast than fast movers, so if two firms in the same industry have different proportions of slow and fast movers, their best achievable accuracy will differ widely - even if they are identical on all other attributes described above (which they will never be).

Better: look at internal benchmarks, e.g., simple forecasts.

See also Kolassa (2008, Foresight), "Can We Obtain Valid Benchmarks from Published Surveys of Forecast Accuracy?" Or take a look at How to know that your machine learning problem is hopeless?

$\endgroup$
2
  • $\begingroup$ But is it wrong to have a benchmark and check, whether we can still improve our model performance by using other algorithms ?. For example for the data set i have..ARIMA is some value but ETS or nnet might give me a different value.. $\endgroup$ Nov 7, 2019 at 20:13
  • $\begingroup$ Yes, but knowing whether other firms in the industry have a lower or a higher MAPE does not tell you anything about whether your MAPE is "good" or "bad". You may have easily forecastable series, so if your MAPE is the same as the "benchmark", you are still not doing the best you can. Or the other way around. $\endgroup$ Nov 7, 2019 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.