Can RMSE and MAE have the same value?

Question

I am implementing cross validation and calculating error metrics such as RMSE, $R^2$, MAE, MSE, etc.

Can RMSE and MAE have the same value?

Answer 1

Yes, in theory. The simplest case I can imagine is a dataset where all the prediction errors (i.e. residuals) are exactly $\pm$1. RMSE and MAE will return identical values of 1. One can construct other scenarios too, but none seem very likely.

EDIT: Thanks to @DilipSarwate for pointing out (further elaborated upon by @user20160 in their excellent answer) that this result is possible if and only if the absolute values of all prediction errors are identical. There is nothing special about the value $\pm$1 in my example, in other words; any other number would work instead of 1.

Answer 2

The mean absolute error (MAE) can equal the mean squared error (MSE) or root mean squared error (RMSE) under certain conditions, which I'll show below. These conditions are unlikely to occur in practice.

Preliminaries

Let $r_i = |y_i - \hat{y}_i|$ denote the absolute value of the residual for the $i$th data point, and let $r = [r_i, \dots, r_n]^T$ be a vector containing absolute residuals for all $n$ points in the dataset. Letting $\vec{1}$ denote a $n \times 1$ vector of ones, the MAE, MSE, and RMSE can be written as:

$$MAE = \frac{1}{n} \vec{1}^T r \quad MSE = \frac{1}{n} r^T r \quad RMSE = \sqrt{\frac{1}{n} r^T r} \tag{1}$$

MSE

Setting the MSE equal to the MAE and rearranging gives:

$$(r - \vec{1})^T r = 0 \tag{2}$$

The MSE and MAE are equal for all datasets where the absolute residuals solve the above equation. Two obvious solutions are: $r = \vec{0}$ (there's zero error) and $r = \vec{1}$ (the residuals are all $\pm 1$, as mkt mentioned). But, there are infinitely many solutions.

We can interpret equation $(2)$ geometrically as follows: The LHS is the dot product of $r-\vec{1}$ and $r$. Zero dot product implies orthogonality. So, the MSE and MAE are equal if subtracting 1 from each absolute residual gives a vector that's orthogonal to the original absolute residuals.

Furthermore, by completing the square, equation $(2)$ can be rewritten as:

$$\Big( r-\frac{1}{2} \vec{1} \Big)^T \Big( r-\frac{1}{2} \vec{1} \Big) = \frac{n}{4} \tag{3}$$

This equation describes an $n$-dimensional sphere centered at $[\frac{1}{2}, \dots, \frac{1}{2}]^T$ with radius $\frac{1}{2} \sqrt{n}$. The MSE and MAE are equal if and only if the absolute residuals lie on the surface of this hypersphere.

RMSE

Setting the RMSE equal to the MAE and rearranging gives:

$$r^T A r = 0 \tag{4}$$

$$A = (n I - \vec{1} \vec{1}^T)$$

where $I$ is the identity matrix. The solution set is the null space of $A$; that is, the set of all $r$ such that $A r = \vec{0}$. To find the null space, note that $A$ is a $n \times n$ matrix with diagonal elements equal to $n-1$ and all other elements equal to $-1$. The statement $A r = \vec{0}$ corresponds to the system of equations:

$$(n-1) r_i - \sum_{j \ne i} r_j = 0 \quad \forall i \tag{5}$$

Or, rearranging things:

$$r_i = \frac{1}{n-1}\sum_{j \ne i} r_j \quad \forall i \tag{6}$$

That is, every element $r_i$ must equal the mean of the other elements. The only way to satisfy this requirement is for all elements to be equal (this result can also be obtained by considering the eigendecomposition of $A$). Therefore, the solution set consists of all nonnegative vectors with identical entries:

$$\{r \mid r = c \vec{1} \enspace \forall c \ge 0\}$$

So, the RMSE and MAE are equal if and only if the absolute values of the residuals are equal for all data points.