3
$\begingroup$

Generally speaking, can RMSE be smaller than MAE?

I am calculating RMSE and MAE for my results. In two out of five methods, the RMSE is smaller than MAE. Note that I am using the same data, the same script, and the same code to calculate RMSE and MAE. The only difference is the algorithm to create the models.

I have been over my script several times and everything is ok with the code.

Thank you.

$\endgroup$
5
  • $\begingroup$ Generally speaking, yes. It depends on algorithms result. Different algorithms - different results. And check your data for outliers, maybe they effect on this measures. $\endgroup$
    – ooolllooo
    Oct 12, 2017 at 11:55
  • $\begingroup$ Can you expand a bit here by editing to say under what circumstances they might vary? $\endgroup$
    – mdewey
    Oct 12, 2017 at 12:21
  • $\begingroup$ For example, true values is 1,2,3 $\endgroup$
    – ooolllooo
    Oct 12, 2017 at 12:39
  • $\begingroup$ Does this answer stats.stackexchange.com/questions/59493/… help you move further with your problem? $\endgroup$
    – mdewey
    Oct 12, 2017 at 12:43
  • 1
    $\begingroup$ No, RMSE is always greater or equal MAE. Take a look at this article and this. $\endgroup$
    – lenhhoxung
    Apr 20, 2019 at 6:22

3 Answers 3

8
$\begingroup$

We should normally have $$MAE \leq RMSE$$

This is a consequence of the triangle inequality (in a similar way as 'square of the mean values < mean of the squared values')

$$ \overbrace{(\vert r_1 \vert + \vert r_2\vert + \dots + \vert r_n \vert)^2}^{MAE^2} \leq \overbrace{\vert r_1 \vert^2 + \vert r_2\vert^2 + \dots + \vert r_n \vert^2}^{RMSE^2}$$

We can derive this more explicitly as following by expressing the absolute value of the error terms $\vert r_i \vert$ as a sum of two components: the mean of the absolute value of the error terms and the variation relative to that mean, $\vert r_i \vert = \mu + \delta_{i}$.

$$\begin{array}{} \overbrace{(r_1^2 + r_2^2 + \dots + r_n^2)}^{n \cdot RMSE^2} &=& \vert r_1 \vert^2 + \vert r_2\vert^2 + \dots + \vert r_3 \vert^2 \\ & = & (\mu+\delta_1)^2 + (\mu+\delta_2)^2 +\dots + (\mu+\delta_n)^2\\ & = & n \mu^2 + 2 \mu (\delta_1 + \delta_2 +\dots + \delta_n) + \delta_1^2 + \delta_2^2 + \dots + \delta_n^2\\ & = & n \mu^2 + \delta_1^2 + \delta_2^2 + \dots + \delta_n^2 \geq \underbrace{n\mu^2}_{n \cdot MAE^2} \end{array}$$

The step where $2 \mu (\delta_1 + \delta_2 +\dots + \delta_n)$ is removed comes from $\overline{\delta_{\vert r_i \vert}}=0$. Note that this must be the case for $\mu$ to be the mean of ${\vert r_i \vert}$, since $\overline{\vert r_i \vert} = \overline{\mu + \delta_i} = \overline{\mu} + \overline{\delta_i} = \mu + \overline{\delta_i}$)

The equality arises when you have all $\delta_i=0$, this is the case when $\vert r_i \vert = \mu$, or when $r_i = \pm \mu$.


Possible exception for different definition of 'mean'

Normally you compute the mean operation in RMSE and MAE by division with $n$ and then you get that the above inequality becomes

$$\frac{(r_1^2 + r_2^2 + \dots + r_n^2)}{n} \geq \mu^2 = \left( \frac{\vert r_1\vert + \vert r_2\vert + \dots + \vert r_n\vert }{n} \right)^2$$

But it is possible that you use a division by $n-1$ or $n-p$. And you compare

$$\frac{(r_1^2 + r_2^2 + \dots + r_n^2)}{n-p} \quad\text{versus} \quad \left( \frac{\vert r_1\vert + \vert r_2\vert + \dots + \vert r_n\vert }{n-p} \right)^2$$

In this case it is possible that the right side is larger than the left side. Maybe this is the case for your code where you mention different methods to compute MAE and RMSE.

Example: let $r_1 = r_2 = 1$ and $r_3 = r_4 = -1$, let $n=1$ and $p=1$, then

$$\begin{array}{rcccl} RMSE &=& \sqrt{\frac{1^2+1^2+(-1)^2+(-1)^2}{3}} &=& \sqrt{\frac{4}{3}} &\approx& 1.1547 \\ MAE &=& \frac{\vert 1 \vert+\vert 1 \vert+\vert -1 \vert+\vert -1 \vert}{3} &=& {\frac{4}{3}} &\approx& 1.3333 \end{array}$$

$\endgroup$
3
$\begingroup$

As a complement to Sextus' answer (+1), here is how I convinced myself that RMSE >= MAE:

We know that $$ \mathbf{Var}\left[f(X)\right] = \mathbf{E}\left[f(X)^2 \right] - \mathbf{E}\left[f(X) \right]^2 \geq 0 $$

If we use this fact with $f$ as the absolute value function and the model's residuals (errors) as our $X$, we get

$$ \mathbf{E}\left[\text{abs}(\text{error})^2 \right] \geq \mathbf{E}\left[\text{abs}(\text{error}) \right]^2 $$ $$ \mathbf{E}\left[\text{error}^2 \right] \geq \text{MAE}^2 $$ $$ \text{MSE} \geq \text{MAE}^2 $$ $$ \text{RMSE} \geq \text{MAE} $$

An example where they are equal: we have errors $(-1, -1, 1, 1)$, and our RMSE and MAE are both equal to 1. This should happen whenever $\mathbf{Var}\left[\text{abs}(\text{error})\right] = 0$. Another example would be errors $(-0.5, -0.5, 0.5)$, in which case our RMSE and MAE are both $0.5$.

$\endgroup$
0
$\begingroup$

In plain English, it is not possible that the MAE > Standard Error. The MAE is essentially fairly close to the Median absolute error (the 50th percentile). The Standard Error is around the 68th percentile. Both statements, assume that the errors in your model are not too far off from being normally distributed. If your model is fairly well specified, it is quite likely.

$\endgroup$
1
  • $\begingroup$ I am not clear why my answer was recently downgraded. I thought a plain English answer was a clarifying alternative to the other formula driven answers. $\endgroup$
    – Sympa
    May 27, 2021 at 19:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.