3
$\begingroup$

What if I have a regression model that predict amount of products that have been sold with rmse 1,5. Can I say, that in each observation error is +- 1,5 items?

$\endgroup$

2 Answers 2

7
$\begingroup$

You could simply compute the mean absolute error between the truth and your predictions, on your test set. Then you could claim something like: with data similarly distributed to the data we have in the test set, we can expect a +- error on average for unseen items.

$\endgroup$
1
  • 7
    $\begingroup$ (+1) MAE is great to convey error to people who are not very comfortable with mathematical concepts. $\endgroup$
    – usεr11852
    Commented Jul 3, 2018 at 20:35
10
$\begingroup$

Definitely not.

For instance, your actual sales might be Poisson distributed with an average of $\lambda=1.5^2=2.25$. In this case, your minimum RMSE point forecast is $1.5^2$ for every future time point:

sales

You can prove mathematically, that the long-run RMSE will be exactly $1.5$. However, every single point error is of course a value such as $-2.25, -1.25, -0.25, 0.75, 1.75, \dots$:

errors

All you can say is that the root mean squared error is $1.5$. Which is, admittedly, not overly useful.

R code:

set.seed(1)
xx <- rpois(1e2,1.5^2)

plot(xx,type="o",pch=19,xlab="Sales",ylab="Time")
abline(h=1.5^2,col="red")

errors <- xx-1.5^2
hist(errors,breaks=seq(-3,5))
$\endgroup$
3
  • $\begingroup$ If you could show your errors were sufficiently close to normal distribution, could you do some interval? Confidence, prediction, tolerance? I guess tolerance would be best for what OP needs? $\endgroup$
    – Mooks
    Commented Jul 4, 2018 at 10:06
  • $\begingroup$ You could. It sounds like the OP would be most interested in a prediction-interval. And of course you can calculate these based on any distributional assumption, also non-normal. $\endgroup$ Commented Jul 4, 2018 at 11:02
  • $\begingroup$ What a dope, I’ve even done it for a log-normal using EnvStats once! Prediction interval? Ah yes, of course! The dependent variable is for the total number of sales - not even sure what the tolerance interval would be saying in that case?! $\endgroup$
    – Mooks
    Commented Jul 4, 2018 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.