# Can there be a situation where one regression model gives lower RMSE than the other but also lower R-squared?

Consider the following scenario where you use the same data X (the same number of predictors p, same number of observations n) to predict a continuous outcome y, in 2 different regression models (e.g. Linear Regression and Random Forests). For both models you calculate RMSE and R-squared (assume simple correlation between y and y-hat, squared).

You get:

RMSE1 < RMSE2

R-squared1 < R-squared2


Could someone explain if this scenario is possible and how it occurs? A simple simulation in R could also do.

This may arise if the regression model is nonlinear. Consider the following true data

0.6
0.55
0.3
0.24
0.22


With model 1 predicting

0.82
0.72
0.55
0.29
0.1


and model 2

0.7
0.58
0.51
0.2
0.09


$R^2$ for Model 1 is 0.8448 while for Model 2 is 0.7866.

However, Model 1 has higher RMS error, at 0.1770 but Model 2 only has RMSE of 0.1212

This happens because $R^2$ is only really valid for linear regression models (and data).