2
$\begingroup$

This is a very basic question but I have trouble making sense of it.

I am reading a paper (Feigin, Micha, Daniel Freedman, and Brian W. Anthony. "A deep learning framework for single-sided sound speed inversion in medical ultrasound." IEEE Transactions on Biomedical Engineering 67.4 (2019): 1142-1151.), the paper is a deep learning paper(a CNN is trained). I came across a table in which the error rates are reported. The error is a rooted mean squared error, plus there is also a mean and standard deviation reported for MAE.

  • Is this common to report mean and standard deviation of MAE?

  • Why would one need to know for example, standard deviation or even mean of MAE? Isn't MAE itself an interpretable metric?

$\endgroup$
5
  • $\begingroup$ Was there some kind of cross validation? $\endgroup$
    – Dave
    Commented Jul 20, 2020 at 9:54
  • $\begingroup$ No I don't think there is $\endgroup$
    – Farnaz
    Commented Jul 20, 2020 at 10:00
  • 3
    $\begingroup$ Please edit your post to give a citation (grab it from Google Scholar) for the paper. Fee free to post a link, too, but links can break, so the full citation is useful. $\endgroup$
    – Dave
    Commented Jul 20, 2020 at 10:04
  • $\begingroup$ The authors provide some motivation for the use of MAE, namely "immunity" to outliers. This might be useful: math.stackexchange.com/questions/1926272/… $\endgroup$
    – Saleh
    Commented Jul 20, 2020 at 11:13
  • $\begingroup$ @Saleh, thanks, but my question is not why MAE is chosen, my question is: what is mean and standard deviation of MAE? do we have mean of mean absolute error? $\endgroup$
    – Farnaz
    Commented Jul 20, 2020 at 11:22

1 Answer 1

5
$\begingroup$

I haven't read the paper but I think they simply report Mean Absolute Error and the standard deviation from it. The MAE measures the average absolute error over the whole dataset. While the standard deviation measures how far the absolute error on each training point from the MAE. A low standard deviation means that errors across all dataset tend to have similar values close to the mean. A high standard deviation tells that the errors are spread over a bigger range.

Why would one want to compute the standard deviation from the MAE?

This can provide some insight into your model. A model with a low Mean Absolute Error tells you that you have a good "average" performance over the whole dataset. If you also have a low standard deviation then you can say that your performance is not good only on average, but also "uniformly" on the whole dataset.

$\endgroup$
3
  • $\begingroup$ Makes sense! but what about mean of mean absolute error? is mean of absolute error over each point different from MAE? $\endgroup$
    – Farnaz
    Commented Jul 20, 2020 at 12:56
  • 1
    $\begingroup$ It makes no sense to me to talk about a mean of the mean absolute error! As far as I can tell, they talk about a mean of mean only once in the description of table III, this can be a typo. I think they just intend saying: mean and standard deviation of the absolute error. $\endgroup$
    – Saleh
    Commented Jul 20, 2020 at 13:03
  • 1
    $\begingroup$ In time series forecasting, one will often calculate an MAE for each series, taking the mean over multiple forecast horizons. And then one may calculate the mean over multiple series' MAEs, or their standard deviation. $\endgroup$ Commented Jul 20, 2020 at 13:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.