Skip to main content
3 of 3
added 1 character in body
user20160
  • 33.2k
  • 3
  • 82
  • 114

The mean absolute error (MAE) can equal the mean squared error (MSE) or root mean squared error (RMSE) under certain conditions, which I'll show below. These conditions are unlikely to occur in practice.

Preliminaries

Let $r_i = |y_i - \hat{y}_i|$ denote the absolute value of the residual for the $i$th data point, and let $r = [r_i, \dots, r_n]^T$ be a vector containing absolute residuals for all $n$ points in the dataset. Letting $\vec{1}$ denote a $n \times 1$ vector of ones, the MAE, MSE, and RMSE can be written as:

$$MAE = \frac{1}{n} \vec{1}^T r \quad MSE = \frac{1}{n} r^T r \quad RMSE = \sqrt{\frac{1}{n} r^T r} \tag{1}$$

MSE

Setting the MSE equal to the MAE and rearranging gives:

$$(r - \vec{1})^T r = 0 \tag{2}$$

The MSE and MAE are equal for all datasets where the absolute residuals solve the above equation. Two obvious solutions are: $r = \vec{0}$ (there's zero error) and $r = \vec{1}$ (the residuals are all $\pm 1$, as mkt mentioned). But, there are infinitely many solutions.

We can interpret equation $(2)$ geometrically as follows: The LHS is the dot product of $r-\vec{1}$ and $r$. Zero dot product implies orthogonality. So, the MSE and MAE are equal if subtracting 1 from each absolute residual gives a vector that's orthogonal to the original absolute residuals.

Furthermore, by completing the square, equation $(2)$ can be rewritten as:

$$\Big( r-\frac{1}{2} \vec{1} \Big)^T \Big( r-\frac{1}{2} \vec{1} \Big) = \frac{n}{4} \tag{3}$$

This equation describes an $n$-dimensional sphere centered at $[\frac{1}{2}, \dots, \frac{1}{2}]^T$ with radius $\frac{1}{2} \sqrt{n}$. The MSE and MAE are equal if and only if the absolute residuals lie on the surface of this hypersphere.

RMSE

Setting the RMSE equal to the MAE and rearranging gives:

$$r^T A r = 0 \tag{4}$$

$$A = (n I - \vec{1} \vec{1}^T)$$

where $I$ is the identity matrix. The solution set is the null space of $A$; that is, the set of all $r$ such that $A r = \vec{0}$. To find the null space, note that $A$ is a $n \times n$ matrix with diagonal elements equal to $n-1$ and all other elements equal to $-1$. The statement $A r = \vec{0}$ corresponds to the system of equations:

$$(n-1) r_i - \sum_{j \ne i} r_j = 0 \quad \forall i \tag{5}$$

Or, rearranging things:

$$r_i = \frac{1}{n-1}\sum_{j \ne i} r_j \quad \forall i \tag{6}$$

That is, every element $r_i$ must equal the mean of the other elements. The only way to satisfy this requirement is for all elements to be equal (this result can also be obtained by considering the eigendecomposition of $A$). Therefore, the solution set consists of all nonnegative vectors with identical entries:

$$\{r \mid r = c \vec{1} \enspace \forall c \ge 0\}$$

So, the RMSE and MAE are equal if and only if the absolute values of the residuals are equal for all data points.

user20160
  • 33.2k
  • 3
  • 82
  • 114