So I am trying to deeply understand the concepts of Residual Standard Error (in addition to RMSE) as well as their usefulness. Here is my current understanding:
RMSE - the standard deviation of the residuals, where residuals are the difference between the predicted y and the observed y
Residual Standard Error - is the standard deviation of the mean of residuals. Essentially if we had multiple samples we could find the mean of the residuals and then we would find the standard deviation of those means. (My understanding of standard errors comes from this StatQuest video)
I guess this is where it breaks down for me. Aren't the mean of residuals always going to be 0? Hence it doesn't make sense to take the standard deviation of those means... Does standard error in this context mean something different from the typical context of standard error?
Follow up - assuming that I can get clarity of residual standard error, when/why is it used, relative to RMSE? Both seem like measurements that would give a sense of the average difference you'd expect to see from a mean if you took another sample. Can someone give a concrete example of: "I use RMSE for/when X and I use residual standard error for/when Y"?
I've already looked at this Stack Exchange post but it was more centered on just the formulas and how to generate them in R. If I could get a simple, explain-like-I'm-five answer that would be greatly appreciated! :)
