How to interpret RMSE for customer?

Question

What if I have a regression model that predict amount of products that have been sold with rmse 1,5. Can I say, that in each observation error is +- 1,5 items?

Answer 1

You could simply compute the mean absolute error between the truth and your predictions, on your test set. Then you could claim something like: with data similarly distributed to the data we have in the test set, we can expect a +- error on average for unseen items.

Answer 2

Definitely not.

For instance, your actual sales might be Poisson distributed with an average of $\lambda=1.5^2=2.25$. In this case, your minimum RMSE point forecast is $1.5^2$ for every future time point:

You can prove mathematically, that the long-run RMSE will be exactly $1.5$. However, every single point error is of course a value such as $-2.25, -1.25, -0.25, 0.75, 1.75, \dots$:

All you can say is that the root mean squared error is $1.5$. Which is, admittedly, not overly useful.

R code:

set.seed(1)
xx <- rpois(1e2,1.5^2)

plot(xx,type="o",pch=19,xlab="Sales",ylab="Time")
abline(h=1.5^2,col="red")

errors <- xx-1.5^2
hist(errors,breaks=seq(-3,5))