9
$\begingroup$

I am implementing cross validation and calculating error metrics such as RMSE, $R^2$, MAE, MSE, etc.

Can RMSE and MAE have the same value?

$\endgroup$
  • 1
    $\begingroup$ Yes. Why not? Let $X$ be always $0$ and a predictor for $X$ be always $1$. There you have it $\endgroup$ – David Aug 26 at 15:41
17
$\begingroup$

Yes, in theory. The simplest case I can imagine is a dataset where all the prediction errors (i.e. residuals) are exactly $\pm$1. RMSE and MAE will return identical values of 1. One can construct other scenarios too, but none seem very likely.

EDIT: Thanks to @DilipSarwate for pointing out (further elaborated upon by @user20160 in their excellent answer) that this result is possible if and only if the absolute values of all prediction errors are identical. There is nothing special about the value $\pm$1 in my example, in other words; any other number would work instead of 1.

$\endgroup$
  • 1
    $\begingroup$ Could you give an example of the other scenarios that you envision? I mean an example other than a scalar multiple (when all the residuals are $\pm \sigma$ instead of $\pm 1$) of the example above. $\endgroup$ – Dilip Sarwate Aug 24 at 17:56
  • $\begingroup$ @DilipSarwate I was pondering this when user20160 added a much better answer that covers it in more detail than I could. $\endgroup$ – mkt Aug 24 at 18:42
  • 1
    $\begingroup$ @mkt Thanks for the kind words. Your answer is correct and concise (+1) $\endgroup$ – user20160 Aug 25 at 0:41
  • $\begingroup$ @DilipSarwate Thanks for the input $\endgroup$ – mkt Aug 25 at 4:49
  • 1
    $\begingroup$ A couple of additional embellishments to your answer: (i) $n$ must be even (say $n=2k$) and (ii) exactly $k$ residuals must have value $+\sigma$ and exactly $k$ residuals must have value $-\sigma$, which of course means that all residuals have absolute value $\sigma$ as you state, but (ii) ensures that the residuals sum to $0$ as they must. The residuals are the deviations from the mean and so must sum to zero. $\endgroup$ – Dilip Sarwate Aug 26 at 22:04
23
$\begingroup$

The mean absolute error (MAE) can equal the mean squared error (MSE) or root mean squared error (RMSE) under certain conditions, which I'll show below. These conditions are unlikely to occur in practice.

Preliminaries

Let $r_i = |y_i - \hat{y}_i|$ denote the absolute value of the residual for the $i$th data point, and let $r = [r_i, \dots, r_n]^T$ be a vector containing absolute residuals for all $n$ points in the dataset. Letting $\vec{1}$ denote a $n \times 1$ vector of ones, the MAE, MSE, and RMSE can be written as:

$$MAE = \frac{1}{n} \vec{1}^T r \quad MSE = \frac{1}{n} r^T r \quad RMSE = \sqrt{\frac{1}{n} r^T r} \tag{1}$$

MSE

Setting the MSE equal to the MAE and rearranging gives:

$$(r - \vec{1})^T r = 0 \tag{2}$$

The MSE and MAE are equal for all datasets where the absolute residuals solve the above equation. Two obvious solutions are: $r = \vec{0}$ (there's zero error) and $r = \vec{1}$ (the residuals are all $\pm 1$, as mkt mentioned). But, there are infinitely many solutions.

We can interpret equation $(2)$ geometrically as follows: The LHS is the dot product of $r-\vec{1}$ and $r$. Zero dot product implies orthogonality. So, the MSE and MAE are equal if subtracting 1 from each absolute residual gives a vector that's orthogonal to the original absolute residuals.

Furthermore, by completing the square, equation $(2)$ can be rewritten as:

$$\Big( r-\frac{1}{2} \vec{1} \Big)^T \Big( r-\frac{1}{2} \vec{1} \Big) = \frac{n}{4} \tag{3}$$

This equation describes an $n$-dimensional sphere centered at $[\frac{1}{2}, \dots, \frac{1}{2}]^T$ with radius $\frac{1}{2} \sqrt{n}$. The MSE and MAE are equal if and only if the absolute residuals lie on the surface of this hypersphere.

RMSE

Setting the RMSE equal to the MAE and rearranging gives:

$$r^T A r = 0 \tag{4}$$

$$A = (n I - \vec{1} \vec{1}^T)$$

where $I$ is the identity matrix. The solution set is the null space of $A$; that is, the set of all $r$ such that $A r = \vec{0}$. To find the null space, note that $A$ is a $n \times n$ matrix with diagonal elements equal to $n-1$ and all other elements equal to $-1$. The statement $A r = \vec{0}$ corresponds to the system of equations:

$$(n-1) r_i - \sum_{j \ne i} r_j = 0 \quad \forall i \tag{5}$$

Or, rearranging things:

$$r_i = \frac{1}{n-1}\sum_{j \ne i} r_j \quad \forall i \tag{6}$$

That is, every element $r_i$ must equal the mean of the other elements. The only way to satisfy this requirement is for all elements to be equal (this result can also be obtained by considering the eigendecomposition of $A$). Therefore, the solution set consists of all nonnegative vectors with identical entries:

$$\{r \mid r = c \vec{1} \enspace \forall c \ge 0\}$$

So, the RMSE and MAE are equal if and only if the absolute values of the residuals are equal for all data points.

$\endgroup$
  • 1
    $\begingroup$ +1. I felt a need to verify that most of this hypersphere lies in the region where all the components of $r$ are non-negative, which is a requirement of absolute residuals: that convinced me there truly are a great many (non-trivial) solutions. $\endgroup$ – whuber Aug 24 at 18:29
  • $\begingroup$ +1 Great answer! $\endgroup$ – mkt Aug 24 at 18:42
  • 1
    $\begingroup$ Actually, the question was whether RMSE and MAE can ever be equal and not whether MSE and MAE can ever be equal. Perhaps @mkt's answer (or the generalized version thereof that I suggested in a comment) is the only answer to the RMSE = MAE question? $\endgroup$ – Dilip Sarwate Aug 24 at 21:10
  • $\begingroup$ @DilipSarwate, Yes, realized after posting this that I had skipped the 'R' part. I've edited to include RMSE now. I believe the version you suggested is the only possible answer in this case. $\endgroup$ – user20160 Aug 25 at 0:28
  • 2
    $\begingroup$ @Hiyam If there is only 1 value, then RMSE by definition must be equal to MAE. Because there is just 1 error, squaring it and taking the root just returns the absolute value of the original error. $\endgroup$ – mkt Aug 25 at 9:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.