# Regression models with comparable MAE but differing R²

I have trained two regression models on the same dataset. They perform with comparable mean absolut errors $MAE_{1,2} \approx 0.45$, but the coefficient of determination differs significantly with $R^2_1 \approx 0$ (well, that's bad!) in one case and $R^2_2 \approx 0.4$ (mhh...) in the other.

I understand that a better R² means the model explains the encountered variance better. But how to interpret this in the case of equal MAE? Model two just has a smaller variance?

NB: I understand that my choice of metrics might be questionable.

• I chose R² instead of adjusted R² simply because model two is a deep neural network and I did not find any literature in how to compute it in this case.
• I chose MAE over mean squared error MSE because the values I predict fall into a well-define range $[0,10]$, which makes MAE intuitively meaningful to me, whereas MSE would be not.

If the behavior I observe is due to this choice of metrics I would be glad to learn how this is the case! Also please excuse if this turns out a noob question, my understanding of statistics is somewhat empirical in its nature.

• A couple of years late, but anyway: the easiest way to understand what's going on would be by plotting the data and model fits. Especially plots of observed vs. predicted values for both models.
– mkt
Commented Aug 13, 2018 at 11:15
• Something important to keep in mind is that R^2 is related to the MSE loss function, not MAE. MSE gives a stronger penalty to large deviations from your prediction.
– Dave
Commented May 12, 2019 at 1:23

$$R^2$$ is a function of MSE loss.
$$R^2=\dfrac{ \sum_i\big( y_i-\hat y_i \big)^2 }{\sum_i\big( y_i-\bar y \big)^2}\\=\dfrac{ nMSE }{\sum_i\big( y_i-\bar y \big)^2}$$
Consequently, if you are comparing two models on their respective $$R^2$$ values, you are implicitly using MSE. Any model with lower (worse) $$R^2$$ than another of the same data has a higher (worse) MSE.
My interpretation of your results than both models have comparable MAE but different MSE is that one (low $$R^2$$) tends to make small mistakes with the occasional terrible prediction, while the other tends to make larger mistakes but fewer of the colossal errors.