58
$\begingroup$

How do I calculate relative error when the true value is zero?

Say I have $x_{true} = 0$ and $x_{test}$. If I define relative error as:

$$\text{relative error} = \frac{x_{true}-x_{test}}{x_{true}}$$

Then the relative error is always undefined. If instead I use the definition:

$$\text{relative error} = \frac{x_{true}-x_{test}}{x_{test}}$$

Then the relative error is always 100%. Both methods seem useless. Is there another alternative?

$\endgroup$
3
  • $\begingroup$ I had the exact same question regarding parameter bias in Monte Carlo simulations, using your first definition. One of my parameter values was 0, so I didn't calculate parameter bias for this particular parameter... $\endgroup$ Commented Feb 15, 2014 at 23:12
  • 3
    $\begingroup$ The solution is to not use relative error in this case. $\endgroup$ Commented Feb 15, 2014 at 23:30
  • $\begingroup$ It can depend what $x$ is. It may make sense to transform via exponential and look at relative error there. $\endgroup$ Commented Sep 22, 2021 at 3:22

4 Answers 4

60
$\begingroup$

There are many alternatives, depending on the purpose.


A common one is the "Relative Percent Difference," or RPD, used in laboratory quality control procedures. Although you can find many seemingly different formulas, they all come down to comparing the difference of two values to their average magnitude:

$$d_1(x,y) = \frac{x - y}{(|x| + |y|)/2} = 2\frac{x - y}{|x| + |y|}.$$

This is a signed expression, positive when $x$ exceeds $y$ and negative when $y$ exceeds $x$. Its value always lies between $-2$ and $2$. By using absolute values in the denominator it handles negative numbers in a reasonable way. Most of the references I can find, such as the New Jersey DEP Site Remediation Program Data Quality Assessment and Data Usability Evaluation Technical Guidance, use the absolute value of $d_1$ because they are interested only in the magnitude of the relative error.


A Wikipedia article on Relative Change and Difference observes that

$$d_\infty(x,y) = \frac{|x - y|}{\max(|x|, |y|)}$$

is frequently used as a relative tolerance test in floating point numerical algorithms. The same article also points out that formulas like $d_1$ and $d_\infty$ may be generalized to

$$d_f(x,y) = \frac{x - y}{f(x,y)}$$

where the function $f$ depends directly on the magnitudes of $x$ and $y$ (usually assuming $x$ and $y$ are positive). As examples it offers their max, min, and arithmetic mean (with and without taking the absolute values of $x$ and $y$ themselves), but one could contemplate other sorts of averages such as the geometric mean $\sqrt{|x y|}$, the harmonic mean $2/(1/|x| + 1/|y|)$ and $L^p$ means $((|x|^p + |y|^p)/2)^{1/p}$. ($d_1$ corresponds to $p=1$ and $d_\infty$ corresponds to the limit as $p\to \infty$.) One might choose an $f$ based on the expected statistical behavior of $x$ and $y$. For instance, with approximately lognormal distributions the geometric mean would be an attractive choice for $f$ because it is a meaningful average in that circumstance.


Most of these formulas run into difficulties when the denominator equals zero. In many applications that either is not possible or it is harmless to set the difference to zero when $x=y=0$.

Note that all these definitions share a fundamental invariance property: whatever the relative difference function $d$ may be, it does not change when the arguments are uniformly rescaled by $\lambda \gt 0$:

$$d(x,y) = d(\lambda x, \lambda y).$$

It is this property that allows us to consider $d$ to be a relative difference. Thus, in particular, a non-invariant function like

$$d(x,y) =?\ \frac{|x-y|}{1 + |y|}$$

simply does not qualify. Whatever virtues it might have, it does not express a relative difference.


The story does not end here. We might even find it fruitful to push the implications of invariance a little further.

The set of all ordered pairs of real numbers $(x,y)\ne (0,0)$ where $(x,y)$ is considered to be the same as $(\lambda x, \lambda y)$ is the Real Projective Line $\mathbb{RP}^1$. In both a topological sense and an algebraic sense, $\mathbb{RP}^1$ is a circle. Any $(x,y)\ne (0,0)$ determines a unique line through the origin $(0,0)$. When $x\ne 0$ its slope is $y/x$; otherwise we may consider its slope to be "infinite" (and either negative or positive). A neighborhood of this vertical line consists of lines with extremely large positive or extremely large negative slopes. We may parameterize all such lines in terms of their angle $\theta = \arctan(y/x)$, with $-\pi/2 \lt \theta \le \pi/2$. Associated with every such $\theta$ is a point on the circle,

$$(\xi, \eta) = (\cos(2\theta), \sin(2\theta)) = \left(\frac{x^2-y^2}{x^2+y^2}, \frac{2xy}{x^2+y^2}\right).$$

Any distance defined on the circle can therefore be used to define a relative difference.

As an example of where this can lead, consider the usual (Euclidean) distance on the circle, whereby the distance between two points is the size of the angle between them. The relative difference is least when $x=y$, corresponding to $2\theta = \pi/2$ (or $2\theta = -3\pi/2$ when $x$ and $y$ have opposite signs). From this point of view a natural relative difference for positive numbers $x$ and $y$ would be the distance to this angle:

$$d_S(x,y) = \left|2\arctan\left(\frac{y}{x}\right) - \pi/2\right|.$$

To first order, this is the relative distance $|x-y|/|y|$--but it works even when $y=0$. Moreover, it doesn't blow up, but instead (as a signed distance) is limited between $-\pi/2$ and $\pi/2$, as this graph indicates:

Figure

This hints at how flexible the choices are when selecting a way to measure relative differences.

$\endgroup$
4
  • $\begingroup$ Thanks for the comprehensive answer, what do you think is the best reference for this line : "is frequently used as a relative tolerance test in floating point numerical algorithms. The same article also points out that formulas like d1d1 and d∞d∞ may be generalized to" $\endgroup$ Commented Jul 20, 2017 at 14:55
  • $\begingroup$ @Hammad Did you follow the link to the Wikipedia article? $\endgroup$
    – whuber
    Commented Jul 20, 2017 at 15:12
  • $\begingroup$ Yeah ! I did take a look on Wikipedia; I think that's not an actual reference ( also that line is without any reference on the wiki ) $\endgroup$ Commented Jul 22, 2017 at 4:25
  • 1
    $\begingroup$ btw, nevermind I found an academic reference for this :) tandfonline.com/doi/abs/10.1080/00031305.1985.10479385 $\endgroup$ Commented Jul 22, 2017 at 4:28
12
$\begingroup$

First, note that you typically take the absolute value in computing the relative error.

A common solution to the problem is to compute

$$\text{relative error}=\frac{\left| x_{\text{true}}- x_{\text{test}} \right|}{1+\left|x_{\text{true}} \right|} .$$

$\endgroup$
7
  • 7
    $\begingroup$ This is problematic in that it varies depending on the units of measure chosen for the values. $\endgroup$
    – whuber
    Commented May 6, 2014 at 17:57
  • 2
    $\begingroup$ That's absolutely true. This isn't a perfect solution to the problem, but it is a common approach that works reasonably well when $x$ is well scaled. $\endgroup$ Commented May 7, 2014 at 1:28
  • 2
    $\begingroup$ Could you elaborate in your answer on what you mean by "well scaled"? For instance, suppose the data arise from calibration of an aqueous chemical measurement system designed for concentrations between $0$ and $0.000001$ moles/liter which can achieve a precision of, say, three significant digits. Your "relative error" would therefore be constantly zero except for obviously erroneous measurements. In light of this, how exactly would you rescale such data? $\endgroup$
    – whuber
    Commented May 7, 2014 at 13:56
  • 1
    $\begingroup$ Your example is one where the variable isn't well scaled. By "well scaled", I mean that that variable is scaled so that it takes on values in a small range (of e.g. a couple of orders of magnitude) near 1. If your variable takes on values over many orders of magnitude than you've got more serious scaling issues and this simple approach isn't going to be adequate. $\endgroup$ Commented May 7, 2014 at 14:13
  • 4
    $\begingroup$ Any reference for this approach? The name of this method? Thank you. $\endgroup$
    – CroCo
    Commented Jul 14, 2017 at 14:46
3
$\begingroup$

MAPE Formula

Finding MAPE,

It is very debatable topic and many opensource contributors have discussed on the above topic. The most efficient approach till now is followed by the developers. Please refer to this PR to know more.

$\endgroup$
0
$\begingroup$

I was a bit confused on this for a while. In the end, its because if you are trying to measure relative error with respect to zero then you are trying to force something that simply does not exist.

If you think about it, you're comparing apples to oranges when you compare relative error to the error measured from zero, because the error measured from zero is equivalent to the measured value (that's why you get 100% error when you divide by the test number).

For example, consider measuring error of gauge pressure (the relative pressure from atmospheric) vs absolute pressure. Say that you use an instrument to measure the gauge pressure at perfect atmospheric conditions, and your device measured atmospheric pressure spot on so that it should record 0% error. Using the equation you provided, and first assuming we used the measured gauge pressure, to calculate relative error: $$ \text{relative error} = \frac{P_{gauge, true}-P_{gauge, test}}{P_{gauge, true}} $$ Then $P_{gauge, true}=0$ and $P_{gauge,test}=0$ and you do not get 0% error, instead it is undefined. That is because the actual percent error should be using the absolute pressure values like this: $$ \text{relative error} = \frac{P_{absolute, true}-P_{absolute, test}}{P_{absolute, true}} $$ Now $P_{absolute, true}=1atm$ and $P_{absolute,test}=1atm$ and you get 0% error. This is the proper application of relative error. The original application that used gauge pressure was more like "relative error of the relative value" which is a different thing than "relative error". You need to convert the gauge pressure to absolute before measuring the relative error.

The solution to your question is to make sure you are dealing with absolute values when measuring relative error, so that zero is not a possibility. Then you are actually getting relative error, and can use that as an uncertainty or a metric of your real percent error. If you must stick with relative values, than you should be using absolute error, because the relative (percent) error will change depending on your reference point.

It's hard to put a concrete definition on 0... "Zero is the integer denoted 0 that, when used as a counting number, means that no objects are present." - Wolfram MathWorld http://mathworld.wolfram.com/Zero.html

Feel free to nit pick, but zero essentially means nothing, it is not there. This is why it does not make sense to use gauge pressure when calculating relative error. Gauge pressure, though useful, assumes there is nothing at atmospheric pressure. We know this is not the case though, because it has an absolute pressure of 1 atm. Thus, the relative error with respect to nothing, just does not exist, it's undefined.

Feel free to argue against this, simply put: any quick fixes, such as adding one to the bottom value, are faulty and not accurate. They can be still be usefully if you are simply trying to minimize error. If you are trying to make accurate measurements of uncertainty though, not so much...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.