I'm testing different models and my general expectation is that models that have a high coefficient of determination should roughly also have a lower error rate (RMSE in that case) than those with a low R² (please correct me if this premise is wrong).

I've defined three models, linear regression, decision trees and random forests:

from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor, RandomForestClassifier
from sklearn.tree import DecisionTreeRegressor

[some code missing here]

def get_rmse(testdata, reg):
    errorsum = 0
    items = 0
    for index, row in testdata.iterrows():
        y = row["Target"]
        X = np.array(row)[:-1].reshape(1, -1)
        y_hat = reg.predict(X)
        errorsum += (y-y_hat.item())**2
        items += 1
    rmse = np.sqrt(errorsum/items)
    return rmse

reg = LinearRegression().fit(X_train, y_train)
score = reg.score(X_test, y_test)
print(f"R²: {score}")
print(f"RMSE: {get_rmse(testdata, reg)}")

dt = DecisionTreeRegressor()
dt.fit(X_train, y_train)
score = dt.score(X_test, y_test)
print(f"R²: {score}")
print(f"RMSE: {get_rmse(testdata, dt)}")

rf = RandomForestRegressor(n_jobs=-1)
rf.fit(X_train, y_train)
score = rf.score(X_test, y_test)
print(f"R²: {score}")
print(f"RMSE: {get_rmse(testdata, rf)}")

The results for my dataset are:

R²: 0.8322231990679154
RMSE: 1.6443917859748052
R²: 0.967768714719696
RMSE: 3.0779040219133758
R²: 0.9772274437532209
RMSE: 2.020003916126851

So it looks like the linear regression model's RMSE is very low but it also has a low R². Why? Shouldn't we expect that a model that makes less mistakes also explains the data better?


sklearn defines score to be the $R^2$:

$$R^2 = 1 - \frac{\text{RSS}}{\text{TSS}}$$

Where $\text{RSS} = n\text{MSE}$ and $\text{TSS} = (n-1)\operatorname{Var}(y)$, the sample variance of $y$.

The RMSE on the other hand is:

$$\text{RMSE} = \sqrt{\text{MSE}}$$

So we can relate one to the other:

$$R^2 = 1 - \frac{\text{RMSE}^2}{\text{TSS}}$$


$$\text{RMSE} = \sqrt{(1-R^2)\text{TSS}}$$

So, if $\text{RMSE}$ increases, $R^2$ decreases and vice versa.

  • $\begingroup$ That's also my expectation but aren't those value comparable over different models? If you compare linear regression and decision trees in my question, the linear regression model has a R² of 0.8 and a RMSE of 1.64. The decision tree has a R² of 0.97 so my expectation would be that the RMSE a decision tree model makes must definitely be lower than the RMSE of the linear regression, so lower than 1.64. $\endgroup$ – Christian Vorhemus Jan 13 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.