16
$\begingroup$

When calculating the $R^2$ value of a linear regression model, should it be calculated on the training dataset, test dataset or both and why?

Furthermore, when calculating $SS_{\text{res}}$ and $SS_{\text{tot}}$ as per the wikipedia article above, should both sums be over the same data set? In other words, if calculating $SS_{\text{res}}$ over the training dataset, does that require that $SS_{\text{tot}}$ also be calculated over the training dataset? (and similarly for the test dataset.)

$\endgroup$
3
  • 7
    $\begingroup$ For the second question, I do not see any reason why you should not calculate both sums over the same dataset. For the first, it will depend on your goals. When you compute R2 on the training data, R2 will tell you something about how much of the variance within your sample is explained by the model, while computing it on the test set tells you something about the predictive quality of your model. $\endgroup$ Commented May 26, 2018 at 15:07
  • 1
    $\begingroup$ For all those data sets you are gonna calculate the $R^2$ value (or some similar value) once or multiple times (multiple times for doing optimization and scanning to find some optimal parameter set used in the fitting). But for the different sets it is done for different reasons. See for instance: stats.stackexchange.com/questions/19048/… $\endgroup$ Commented May 27, 2018 at 19:23
  • $\begingroup$ Are you running a predictive model? $R^2$ measures the ratio of residual to total variance, mean squared error may be more appropriate for a predictive model. $\endgroup$
    – RobertF
    Commented Apr 28, 2023 at 12:30

3 Answers 3

15
$\begingroup$

The test data shows you how well your model has generalized. When you run the test data through your model, it is the moment you've been waiting for: is it good enough?

In the machine learning world, it is very common to present all of the train, validation and the test metrics, but it is the test accuracy that is the most important.

However, if you get a low $R^2$ score on one, and not the other, then something is off! E.g. If the $R^2_{\text{test}}\ll R^2_{\text{training}}$, then it indicates that your model does not generalize well. That is, if e.g. your test set only contains "unseen" data points, then your model would not appear to extrapolate well (aka a form of covariate shift).

In conclusion: you should compare them! However, in many cases, it's the test-set results you're most interested in.

$\endgroup$
7
  • $\begingroup$ This is pretty hard to arrange in practice in something like Tensorflow :( $\endgroup$
    – rjurney
    Commented Nov 7, 2020 at 17:57
  • 4
    $\begingroup$ What specifically do you find hard about it? Maybe open a separate question about it? $\endgroup$ Commented Nov 13, 2020 at 12:52
  • $\begingroup$ I’ve implemented adjusted R squared for my model as a metric in Tensorflow, but I’m not aware how to pass different metrics for train and test set metrics and it takes the x and y shapes as parameters. I found r squared itself to actually be harmful in modern machine learning with lots of records and features. It tends to be very high even when performance is very poor. $\endgroup$
    – rjurney
    Commented Nov 14, 2020 at 20:08
  • 1
    $\begingroup$ @rjurney It's important, as with any metric, to be sure you're using it to measure the right thing. If R2 is high, then model performance is high according to, well, the criteria R2 has. If you get a high R2 score on a "low performing model" then I guess it's not the R2 criteria you want to evaluate the model on? If you have a metric that suggests that it is low performing, then that's is the metric you should use, as that's how you ultimately evaluate? R2 gives you exactly what it's designed for, and is not a generalized metric that universally tells you whether your model performs well. $\endgroup$ Commented Nov 15, 2020 at 8:51
  • $\begingroup$ Right, but I never found anything saying, “R Squared isn’t appropriate for lots of data or lots of features.” It should always be presented in that light because that is a very common case in which it doesn’t indicate performance whatsoever. $\endgroup$
    – rjurney
    Commented Nov 18, 2020 at 2:27
9
$\begingroup$

When calculating the $R^2$ value of a linear regression model, should it be calculated on the training dataset, test dataset or both and why?

The usual $R^2$ is a fitting measure and must be calculated on the training set. In some regression analysis there is no split in vs out of sample and "in sample = all data".

Furthermore, when calculating $SS_{\text{res}}$ and $SS_{\text{tot}}$ as per the wikipedia article above, should both sums be over the same data set?

Yes, of course.You have three object: residual sum of square, total sum of square, explained sum of square. All of them are computed "in sample".

If you are interested in prediction accuracy, to use in sample measures, as the usual $R^2$, is not a good idea (quite common mistake in the past). You need the so called out of sample $R^2$. Read here: How to calculate out of sample R squared?

$\endgroup$
1
$\begingroup$

An elaboration of the above answer on why it's not a good idea to calculate $R^2$ on test data, different than learning data.

To measure "predictive power" of model, how good it performs on data outside of learning dataset, one should use $R^2_{oos}$ instead of $R^2$. OOS stands from "out of sample".

In $R^2_{oos}$ in denominator we replace $ \Sigma (y - \bar{y}_{test})^2 $ by $ \Sigma (y - \bar{y}_{train})^2 $

If you want to know exactly what happens if one ignores $R^2_{oos}$ and uses $R^2$ on test dataset, read below.

I discovered, to my surprise, when the target variable has high variance compared to "signal" (dependency on feature), then calculating $R^2$ on test dataset (different from learning dataset) will produce negative $R^2$ with guarantee.

Below I put jupyter notebook code in Pyhton so anyone can reproduce it and see it themeselves:

import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
import seaborn as sns

x = np.linspace(-1, 1, num=1_000_000)
X = x.reshape(-1, 1)

# notice 0.05 << 1 (variance of y)
y = np.random.normal(x * 0.05)
df = pd.DataFrame({'X': pd.Series(x), 'y': pd.Series(y)})

ax = sns.histplot(data=df, x='X', y='y', bins=(200, 100))
ax.figure.set_figwidth(18)
ax.figure.set_figheight(9)
ax.grid()
plt.show()

2D histogram of dependency y vs X

from sklearn import ensemble
from sklearn.model_selection import cross_val_score

fraction=0.0001
reg = ensemble.ExtraTreesRegressor(
  n_estimators=20, min_samples_split=fraction * 2, min_samples_leaf=fraction
)
_ = reg.fit(X, y)

print(f'r2 score on learn dataset: {reg.score(X, y)}')
print('Notice above, r2 calculated on learn dataset is positive')

X_pred = np.linspace(-1, 1, num=100)
y_pred = reg.predict(X_pred.reshape(-1, 1))
plt.plot(X_pred, y_pred)
plt.gca().grid()
plt.gca().set_title('Model has correctly captured the trend')
plt.show()

r2 score on learn dataset: 0.0049158435364208275

Notice above, r2 calculated on learn dataset is positive

Model predictions

scores = cross_val_score(reg, X, y, scoring='r2')
print(f'r2 {scores.mean():.4f} ± {scores.std():.4f}')

r2 -0.0023 ± 0.0028

Despite model correctly capturing the trend, cross-validation consistently produces negative r2 on test dataset, different from learning dataset.

UPDATE 2022-01-19

The example I presented above has a technical error, the actual reason for negative $R^2$ is lack of shuffling of $X$, $y$ before cross-validating. Still, the point is correct, after shuffling one can still get negative $R^2$ reliably, and this is still fixed by using $R^2_{oos}$ instead. See Corrected example at github.

UPDATE 2023-04-28

Finally, a good article with detailed study of $R^2_{oos}$

https://arxiv.org/pdf/2302.05131.pdf

$\endgroup$
5
  • $\begingroup$ I'm really struggling to understand what this simulation aims to show. Could you please clarify? $\endgroup$
    – Dave
    Commented Nov 14, 2023 at 21:37
  • $\begingroup$ It demonstrates how under strong noise, usual implementation of $R^2$, which is what most libraries (e.g. scikit-learn) offer, can reliably output negative value, even if the model is good, which misleads one to believe the model has no predictive power. If still unclear, there are some more details in github.com/NikolayXHD/notes/blob/master/mess/… $\endgroup$ Commented Nov 16, 2023 at 1:26
  • $\begingroup$ How is the model good if its mean squared error is lower than that of a model that always predicts the same value no matter what? That's what the sklearn function being $<0$ means. $//$ I $100\%$ support that $R^2_{oos}$ formula with $\bar y_{train}$, however. $\endgroup$
    – Dave
    Commented Nov 16, 2023 at 1:28
  • $\begingroup$ "How is the model good if its mean squared error is lower than that of a model that always predicts the same value no matter what?" This is explained in details in then notebook in the link from previous comment $\endgroup$ Commented Nov 16, 2023 at 1:37
  • $\begingroup$ Higher MSE, not lower. $\endgroup$
    – Dave
    Commented Nov 16, 2023 at 1:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.