# R2 on out-sample data set

The conventional definition of $$R^2$$ is: $$R^2 = 1-SSE/SST$$, where SSE denotes sum of squared errors and SST is total sum of squares ($$n\times variance$$, n being number of sample points in train set).

However, I want to see the fitness of my model on out-of-sample set (test set) or both train and test sets combined. Is it fine to use the same definition of $$R^2$$ by taking SSE and SST appropriately over test set or (train+test) sets, respectively? Eg. I train my model on $$n$$ sample points. I want to check its performance on ($$n+p$$) points ($$p$$ being some new sample points out of the train set). Can I use $$R^2 = 1-(SSE$$ on (n+p) points$$)/(SST$$ on (n+p) points$$)$$?

It makes sense. It is more common however to keep the training and test sets separated. So that you train your model on the train set, and then predict on the test set alone. From there you can calculate the prediction error, and a $$R^2_{pred}$$ if you like. (train on $$n$$ data points, evaluate on $$p$$ data points, in your terms.)

You can also look up stuff like the PRESS statistic, and other cross validation methods.

• It's especially recommended to keep separate training and test (and validation) sets to avoid overfitting and overoptimistic measure of model fit. – chl Oct 17 '20 at 18:42

It makes sense to apply $$R^2= 1-{\sum(y_i-\hat y_i)^2}/{\sum(y_i-\bar y)^2}$$ to test set directly. It's a measure of the size of squared residuals compared to the variance of true values.

Alternatively, if you adopt the notion of deviance (see this answer), then you might use the null model from training data instead:

$$\tilde R^2= 1-\frac{\sum(y_i-\hat y_i)^2}{\sum(y_i-\bar y_{train})^2}$$

I've seen both in use, and both can be justified.