I'm reading a paper about a regression algorithm and I came across a definition of relative error (p. 15). The authors define relative error as

$$RE = \frac{MAD}{(1/T) \Sigma_i^T |y_i - median(y)|}$$ where MAD is the mean absolute distance

$$MAD= \frac{\Sigma_i^T |y_i - \hat y_i|}{T}$$

And $T$ is the number of test instances (conventionaly designated as $n$).

I get that this formula is used to yield a measure of error that is relative to the size of the value being estimated, which is useful for comparing error across datasets with target values of different magnitudes. What I don't know is why we wouldn't just do

$$\frac{\Sigma_i^T (|y_i - \hat y_i|/| y_i|)}{T}$$ and if not, why we use the median instead of the mean.

  • $\begingroup$ I don't think this formula measures the error relative to the size of the value being estimated. It looks more like an $R^2$-like quantity, which measures the proportion of the variability explained by a model compared to the total variability of the data. $\endgroup$ – Aniko Jun 29 '12 at 18:30
  • $\begingroup$ @Aniko I don't know how that would make sense given the context. The authors test their regression algorithm on many different datasets, and then calculate the RE, as per above, for each dataset. Then they average RE across all problems. $\endgroup$ – Matt Munson Jun 29 '12 at 18:59

The median is used because it is a more robust estimate of the center of the distribution. The mean can be heavily influenced by extreme values. The reason that $|y_i|$ is not used is because it does not represent a deviation for a central point.

  • $\begingroup$ I like your answer but I'm still a bit confused. Wikipedia gives a definition of relative error as 'the absolute error divided by the magnitude of the exact value'. It also says 'RE is often used to compare approximations of numbers of widely differing size'. Given this, it seems that the formula I provided would be workable for the purpose of making error comparable across datasets. Could you maybe exemplify why deviation from the central point is important here? $\endgroup$ – Matt Munson Jun 29 '12 at 19:09
  • $\begingroup$ It is suppose to be a relative error. The magnitude of yi is not the error in yi. The coefficient of variation is another parameter with some similarity to this. It is the ratio of the standard devaition to the mean when the mean is positive. It does not involve an error term in the denominator but it is not ameasure of error. $\endgroup$ – Michael R. Chernick Jun 29 '12 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.