Residuals and errors are related but not exchangeable. In Wikipedia I read:
In statistics and optimization, statistical errors and residuals are two closely related and easily confused measures of the deviation of an observed value of an element of a statistical sample from its "theoretical value". The error of an observed value is the deviation of the observed value from the (unobservable) true function value, while the residual of an observed value is the difference between the observed value and the estimated function value.
I would agree with this - but only if we look at it from a frequentist perspective. That is, we have to know the one true population value in order to be able to talk about statistical errors. Now, as far as I understand, in the Bayesian framework there are no such things as true values.
What does that mean for the use of the term "error"? In principle one could probably still talk about "errors" and assume that they, themselves have another underlying distribution but somehow it seems not correct to me to use the above definition in a framework without true values?
Any thoughts on this?