4
$\begingroup$

I have trained two regression models on the same dataset. They perform with comparable mean absolut errors $MAE_{1,2} \approx 0.45$, but the coefficient of determination differs significantly with $R^2_1 \approx 0$ (well, that's bad!) in one case and $R^2_2 \approx 0.4$ (mhh...) in the other.

I understand that a better R² means the model explains the encountered variance better. But how to interpret this in the case of equal MAE? Model two just has a smaller variance?

NB: I understand that my choice of metrics might be questionable.

  • I chose R² instead of adjusted R² simply because model two is a deep neural network and I did not find any literature in how to compute it in this case.
  • I chose MAE over mean squared error MSE because the values I predict fall into a well-define range $[0,10]$, which makes MAE intuitively meaningful to me, whereas MSE would be not.

If the behavior I observe is due to this choice of metrics I would be glad to learn how this is the case! Also please excuse if this turns out a noob question, my understanding of statistics is somewhat empirical in its nature.

$\endgroup$
  • $\begingroup$ A couple of years late, but anyway: the easiest way to understand what's going on would be by plotting the data and model fits. Especially plots of observed vs. predicted values for both models. $\endgroup$ – mkt - Reinstate Monica Aug 13 '18 at 11:15
  • 1
    $\begingroup$ Something important to keep in mind is that R^2 is related to the MSE loss function, not MAE. MSE gives a stronger penalty to large deviations from your prediction. $\endgroup$ – Dave May 12 at 1:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.