Is it incorrect to take the standard deviation of absolute values

Question

Imagine I have a model that predicts a person’s height from a photo (just an example). Each individual has their ground-truth height (H), and the model outputs a predicted height (h). The model error is then error = h - H, and can be either positive or negative. One possible measure of model performance would be the standard deviation of these errors. Another possible measure would be the mean absolute deviation. However I’ve also seen people compute the standard deviation of the absolute errors, and this feels wrong to me. The distribution of absolute errors is not centred on zero, and doesn’t look like a normal distribution at all (it’s more like a half-normal distribution).

Am I correct in thinking that we shouldn’t compute the standard deviation of absolute values?

Answer 1

You can calculate the standard deviation of whatever you want, the normal distribution has nothing to do with it. Standard deviation is the square root of the mean squared error. If your errors were centered at zero, you would be calculating

$$ \sqrt{\frac{1}{N} \sum_{i=1}^N |e_i|^2} $$

that would be the same as the root mean squared error (RMSE) since the square function is insensitive to the sign. If they are not centered at zero, there's nothing less correct in calculating the standard deviation of absolute vs raw errors, as it's just a statistic that can be calculated from any values. The function of a random variable is a random variable and you can calculate standard deviations of random variables.

That said, I can't see what could be the value of calculating it or the interpretation of such a statistic.